From patchwork Tue Sep 29 17:32:25 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "James (Fei) Liu-SSI" X-Patchwork-Id: 7288801 Return-Path: X-Original-To: patchwork-ceph-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 156A5BEEA4 for ; Tue, 29 Sep 2015 17:32:41 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 7C8B82051F for ; Tue, 29 Sep 2015 17:32:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 65AF2206B0 for ; Tue, 29 Sep 2015 17:32:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S935222AbbI2Rce (ORCPT ); Tue, 29 Sep 2015 13:32:34 -0400 Received: from mx2.ssi.samsung.com ([207.140.43.206]:38386 "EHLO mx2.ssi.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932631AbbI2Rc1 (ORCPT ); Tue, 29 Sep 2015 13:32:27 -0400 X-ASG-Debug-ID: 1443547946-0a423a7df8cb150001-RKzwh1 Received: from ssiexch-casht2.ssi.samsung.com ([105.128.134.172]) by mx2.ssi.samsung.com with ESMTP id hB9mBaxoOWFSdx01; Tue, 29 Sep 2015 10:32:26 -0700 (PDT) X-Barracuda-Envelope-From: james.liu@ssi.samsung.com X-Barracuda-Apparent-Source-IP: 105.128.134.172 X-ASG-Whitelist: Client Received: from SSIEXCH-MB3.ssi.samsung.com ([fe80::9dae:55f:7824:a7d1]) by ssiexch-casht2.ssi.samsung.com ([fe80::b5ff:5921:7ac7:4da4%11]) with mapi id 14.03.0123.003; Tue, 29 Sep 2015 10:32:25 -0700 From: "James (Fei) Liu-SSI" To: Varada Kari , Sage Weil , "Matt W. Benjamin" , Loic Dachary CC: ceph-devel Subject: RE: loadable objectstore Thread-Topic: loadable objectstore X-ASG-Orig-Subj: RE: loadable objectstore Thread-Index: AQHQrP0XIpZxx0RAg0655PS0DNdybZ25GzoAgAXu8wCAC0J+gIBtx80AgBrFyQCAAOs1gIAAieAAgACAoAD//4sNUA== Date: Tue, 29 Sep 2015 17:32:25 +0000 Message-ID: <99767EA2E27DD44DB4E9F9B9ACA458C03BD7F709@SSIEXCH-MB3.ssi.samsung.com> References: <1615100301.29.1434984131967.JavaMail.root@thunderbeast.private.linuxbox.com> <46879752.37.1434985621131.JavaMail.root@thunderbeast.private.linuxbox.com> <99767EA2E27DD44DB4E9F9B9ACA458C03BD7F3E5@SSIEXCH-MB3.ssi.samsung.com> <99767EA2E27DD44DB4E9F9B9ACA458C03BD7F68B@SSIEXCH-MB3.ssi.samsung.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [105.128.238.79] MIME-Version: 1.0 X-CFilter-Loop: Reflected X-Barracuda-Connect: UNKNOWN[105.128.134.172] X-Barracuda-Start-Time: 1443547946 X-Barracuda-URL: https://mx1.ssi.samsung.com:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at ssi.samsung.com X-Barracuda-BRTS-Status: 1 Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Hi Varada, I tried to reproduce the error you faced over her and compiled your wip-plugin branch from https://github.com/varadakari/ceph/ and always have two errors with autotools: 1. There is no ceph_ver.h file under ./src directory. I manually generated one and work around the error. 2. gmock error even though I tried to run ./autogen many times. make[3]: *** No rule to make target `../src/gmock/lib/libgmock_main.la', needed by `unittest_librbd'. Stop. make[3]: *** Waiting for unfinished jobs.... make[3]: Leaving directory `/home/jamesliu/WorkSpace/ceph_varada/src' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/home/jamesliu/WorkSpace/ceph_varada/src' make[1]: *** [all] Error 2 make[1]: Leaving directory `/home/jamesliu/WorkSpace/ceph_varada/src' make: *** [all-recursive] Error 1 By the way, CMake won't work either in your repo. thanks, James -----Original Message----- From: Varada Kari [mailto:Varada.Kari@sandisk.com] Sent: Tuesday, September 29, 2015 10:14 AM To: James (Fei) Liu-SSI; Sage Weil; Matt W. Benjamin; Loic Dachary Cc: ceph-devel Subject: RE: loadable objectstore Hi James, Symbol is present in bss. This is what I find $ nm libceph_filestore.so | grep tracepoint_dlopen 000000000047f8c0 b tracepoint_dlopen Please note you have to comment the following line to check all the symbols on the so. Varada > -----Original Message----- > From: James (Fei) Liu-SSI [mailto:james.liu@ssi.samsung.com] > Sent: Tuesday, September 29, 2015 10:08 PM > To: Varada Kari ; Sage Weil > ; Matt W. Benjamin ; Loic > Dachary > Cc: ceph-devel > Subject: RE: loadable objectstore > > Hi Varada, > Can you use command nm to see whether " tracepoint_dlopen" is > defined in "./.libs/libceph_filestore.so"? > > > Regards, > James > > > -----Original Message----- > From: Varada Kari [mailto:Varada.Kari@sandisk.com] > Sent: Monday, September 28, 2015 6:21 PM > To: James (Fei) Liu-SSI; Sage Weil; Matt W. Benjamin; Loic Dachary > Cc: ceph-devel > Subject: RE: loadable objectstore > > No James, Facing library issues with libnss3 and libcommon(ceph). Will > resolve them, generate a new pull request soon on master. > > Thanks, > Varada > > > -----Original Message----- > > From: James (Fei) Liu-SSI [mailto:james.liu@ssi.samsung.com] > > Sent: Monday, September 28, 2015 11:51 PM > > To: Varada Kari ; Sage Weil > > ; Matt W. Benjamin ; Loic > > Dachary > > Cc: ceph-devel > > Subject: RE: loadable objectstore > > > > Hi Varada, > > Have you rebased the pull request to master already? > > > > Thanks, > > James > > > > -----Original Message----- > > From: ceph-devel-owner@vger.kernel.org [mailto:ceph-devel- > > owner@vger.kernel.org] On Behalf Of Varada Kari > > Sent: Friday, September 11, 2015 3:28 AM > > To: Sage Weil; Matt W. Benjamin; Loic Dachary > > Cc: ceph-devel > > Subject: RE: loadable objectstore > > > > Hi Sage/ Matt, > > > > I have submitted the pull request based on wip-plugin branch for the > > object store factory implementation at > https://github.com/ceph/ceph/pull/5884 . > > Haven't rebased to the master yet. Working on rebase and including > > new store in the factory implementation. Please have a look and let > > me know your comments. Will submit a rebased PR soon with new store > integration. > > > > Thanks, > > Varada > > > > -----Original Message----- > > From: ceph-devel-owner@vger.kernel.org [mailto:ceph-devel- > > owner@vger.kernel.org] On Behalf Of Varada Kari > > Sent: Friday, July 03, 2015 7:31 PM > > To: Sage Weil ; Adam Crume > > > Cc: Loic Dachary ; ceph-devel > devel@vger.kernel.org>; Matt W. Benjamin > > Subject: RE: loadable objectstore > > > > Hi All, > > > > Not able to make much progress after making common as a shared > > object along with object store. > > Compilation of the test binaries are failing with "./.libs/libceph_filestore.so: > > undefined reference to `tracepoint_dlopen'". > > > > CXXLD ceph_streamtest > > ./.libs/libceph_filestore.so: undefined reference to `tracepoint_dlopen' > > collect2: error: ld returned 1 exit status > > make[3]: *** [ceph_streamtest] Error 1 > > > > But libfilestore.so is linked with lttng-ust. > > > > src/.libs$ ldd libceph_filestore.so > > libceph_keyvaluestore.so.1 => > > /home/varada/obs-factory/plugin- > > work/src/.libs/libceph_keyvaluestore.so.1 (0x00007f5e50f50000) > > libceph_os.so.1 => /home/varada/obs-factory/plugin- > > work/src/.libs/libceph_os.so.1 (0x00007f5e4f93a000) > > libcommon.so.1 => /home/varada/ obs-factory/plugin- > > work/src/.libs/libcommon.so.1 (0x00007f5e4b5df000) > > liblttng-ust.so.0 => > > /usr/lib/x86_64-linux-gnu/liblttng-ust.so.0 > > (0x00007f5e4b179000) > > liblttng-ust-tracepoint.so.0 => > > /usr/lib/x86_64-linux-gnu/liblttng-ust- > > tracepoint.so.0 (0x00007f5e4a021000) > > liburcu-bp.so.1 => /usr/lib/liburcu-bp.so.1 (0x00007f5e49e1a000) > > liburcu-cds.so.1 => /usr/lib/liburcu-cds.so.1 > > (0x00007f5e49c12000) > > > > Edited the above output just show the dependencies. > > Did anyone face this issue before? > > Any help would be much appreciated. > > > > Thanks, > > Varada > > > > -----Original Message----- > > From: ceph-devel-owner@vger.kernel.org [mailto:ceph-devel- > > owner@vger.kernel.org] On Behalf Of Varada Kari > > Sent: Friday, June 26, 2015 3:34 PM > > To: Sage Weil > > Cc: Loic Dachary; ceph-devel; Matt W. Benjamin > > Subject: RE: loadable objectstore > > > > Hi, > > > > Made some more changes to resolve lttng problems at > > https://github.com/varadakari/ceph/commits/wip-plugin. > > But couldn’t by pass the issues. Facing some issues like mentioned below. > > > > ./.libs/libceph_filestore.so: undefined reference to `tracepoint_dlopen' > > > > Compiling with -llttng-ust is not resolving the problem. Seen some > > threads in devel list before, mentioning this problem. > > Can anyone take a look and guide me to fix this problem? > > > > Haven't made the changes to change the plugin name etc... will be > > making them as part of cleanup. > > > > Thanks, > > Varada > > > > -----Original Message----- > > From: ceph-devel-owner@vger.kernel.org [mailto:ceph-devel- > > owner@vger.kernel.org] On Behalf Of Varada Kari > > Sent: Monday, June 22, 2015 8:57 PM > > To: Matt W. Benjamin > > Cc: Loic Dachary; ceph-devel; Sage Weil > > Subject: RE: loadable objectstore > > > > Hi Matt, > > > > Majority of the changes are segregating the files to corresponding > > shared object and creating a factory object. And the naming is > > mostly taken from Erasure-coding plugins. Want a good naming > > convention :-), hence a preliminary review. Do agree, we have lot of > > loadable interfaces, and I think we are in the way of making them > > on-demand (if > > possible) loadable modules. > > > > Varada > > > > -----Original Message----- > > From: Matt W. Benjamin [mailto:matt@cohortfs.com] > > Sent: Monday, June 22, 2015 8:37 PM > > To: Varada Kari > > Cc: Loic Dachary; ceph-devel; Sage Weil > > Subject: Re: loadable objectstore > > > > Hi, > > > > It's just aesthetic, but it feels clunky to change the names of well > > known modules to Plugin--esp. if that generalizes forward > > to new loadable modules (and we have a lot of loadable interfaces). > > > > Matt > > > > ----- "Varada Kari" wrote: > > > > > Hi Sage, > > > > > > Please find the initial implementation of objects store factory > > > (initial cut) at > > > > > > https://github.com/varadakari/ceph/commit/9d5fe2fecf38ba106c7c7b7a3ed > > e > > > 4f189ec7e1c8 > > > > > > This is still work in progress branch. Right now I am facing Lttng > > > issues, > > > LTTng-UST: Error (-17) while registering tracepoint probe. > > > Duplicate registration of tracepoint probes having the same name is not allowed. > > > > > > Might be an issue with libcommon inclusion. Trying resolving the > > > issue now. Seems I need to make libcommon also as a shared object > > > to avoid the duplicates, static linking is a problem here. > > > Any suggestions or comments on this problem? > > > > > > > > > I have commented out test binary (ceph_test_keyvaluedb_atomicity) > > > compilation, due to unresolved symbols for g_ceph_context and g_conf. > > > Not able to fix/workaround the problem so far. > > > > > > Can you please review if this is what in your mind for the factory > > > implementation? > > > > > > We can extend this implementation to messenger and different > > > backend of the OSD. Will make those incremental once the base > > > framework is approved. > > > > > > Thanks, > > > Varada > > > > > > > > > > > > -----Original Message----- > > > From: Sage Weil [mailto:sage@newdream.net] > > > Sent: Tuesday, May 12, 2015 5:11 AM > > > To: Varada Kari > > > Cc: Loic Dachary; Matt W. Benjamin; ceph-devel > > > Subject: RE: loadable objectstore > > > > > > Hi Varada, > > > > > > Any progress on this? > > > > > > Thanks! > > > sage > > > > > > On Sat, 11 Apr 2015, Varada Kari wrote: > > > > > > > Thanks Sage and Loic. Will add the missing pieces and integrate > > > > the > > > object factory and plugin changes. > > > > > > > > Varada > > > > > > > > -----Original Message----- > > > > From: Sage Weil [mailto:sage@newdream.net] > > > > Sent: Saturday, April 11, 2015 9:19 PM > > > > To: Loic Dachary > > > > Cc: Varada Kari; Matt W. Benjamin; ceph-devel > > > > Subject: Re: loadable objectstore > > > > > > > > On Sat, 11 Apr 2015, Loic Dachary wrote: > > > > > Hi Sage, > > > > > > > > > > It looks like common/PluginRegistry.{h,cc} are missing from > > > > > 18ad8df1094db52c839dc6b2dc689fc882230acb > > > > > > > > whoops pushed! :) > > > > > > > > sage > > > > > > > > > > > > > > Cheers > > > > > > > > > > On 11/04/2015 16:51, Sage Weil wrote: > > > > > > On Sat, 11 Apr 2015, Varada Kari wrote: > > > > > >> Hi Matt/ Sage, > > > > > >> > > > > > >> Thanks for sharing the pull request. Will work on the items > > > what > > > > > >> sage was mentioning in the comments along with the items in > > > > > >> the > > > tracker. > > > > > > > > > > > > I pushed a wip-plugin branch with the pattern I had in mind... > > > > > > does this make sense? It's only compile tested, and still > > > needs > > > > > > preload() methods implemented so that long-running daemons > > > > > > (or perhaps others?) can preload it all at once. > > > > > > > > > > > > Ideally we would replace ErasurePluginRegistry with this > > > > > > too, which means the naming of the .so's needs to be a bit > > > > > > more flexible as the EC plugin scheme is something like > > > > > > /usr/lib/ceph/erasure-code/libec_$name.so (not a strict > > > > > > function > > > > > > > > > of type and name). Or actually, we can just rename the EC > > > > > > .so's > > > > > > > > > when this goes in? Since they're explicitly incompatible > > > between > > > > > > versions there is no reason the names need to remain > > > consistent... > > > > > > > > > > > > sage > > > > > > > > > > > >> > > > > > >> Varada > > > > > >> > > > > > >> -----Original Message----- > > > > > >> From: Sage Weil [mailto:sage@newdream.net] > > > > > >> Sent: Saturday, April 11, 2015 4:16 AM > > > > > >> To: Matt W. Benjamin > > > > > >> Cc: Varada Kari; ceph-devel > > > > > >> Subject: Re: loadable objectstore > > > > > >> > > > > > >> On Fri, 10 Apr 2015, Matt W. Benjamin wrote: > > > > > >>> Hi Varada, > > > > > >>> > > > > > >>> I pushed branch hammer-osfactory to > > > > > >>> https://github.com/linuxbox2/ceph.git > > > > > >>> > > > > > >>> It can at least provide a starting point for discussion, > > > > > >>> if > > > not > > > > > >>> a jumping off point; I do think it's basically what Sage > > > described. > > > > > >> > > > > > >> I made a few comments on the pull request. I think a > > > > > >> factory > > > class is the way to go, but we should be able to make the plugin > > > loading part generic and not interface specific at all... > > > > > >> > > > > > >> sage > > > > > >> > > > > > >>> > > > > > >>> I was able to update the CMake build system to build the > > > > > >>> tree, > > > I > > > > > >>> haven't attempted Autotools. :( > > > > > >>> > > > > > >>> Regards, > > > > > >>> > > > > > >>> Matt > > > > > >>> > > > > > >>> -- > > > > > >>> Matt Benjamin > > > > > >>> CohortFS, LLC. > > > > > >>> 315 West Huron Street, Suite 140A Ann Arbor, Michigan > > > > > >>> 48103 > > > > > >>> > > > > > >>> http://cohortfs.com > > > > > >>> > > > > > >>> tel. 734-761-4689 > > > > > >>> fax. 734-769-8938 > > > > > >>> cel. 734-216-5309 > > > > > >>> -- > > > > > >>> To unsubscribe from this list: send the line "unsubscribe > > > ceph-devel" > > > > > >>> in the body of a message to majordomo@vger.kernel.org More > > > > > >>> majordomo info at > > > > > >>> http://vger.kernel.org/majordomo-info.html > > > > > >>> > > > > > >>> > > > > > >> > > > > > >> ________________________________ > > > > > >> > > > > > >> PLEASE NOTE: The information contained in this electronic > > > > > >> mail > > > message is intended only for the use of the designated > > > recipient(s) named above. If the reader of this message is not the > > > intended recipient, you are hereby notified that you have received > > > this message in error and that any review, dissemination, > > > distribution, or copying of this message is strictly prohibited. > > > If you have received this communication in error, please notify > > > the sender by telephone or e-mail (as shown above) immediately and > > > destroy any and all copies of this message in your possession > > > (whether hard copies or electronically stored copies). > > > > > >> > > > > > >> -- > > > > > >> To unsubscribe from this list: send the line "unsubscribe > > > > > >> ceph-devel" in the body of a message to > > > majordomo@vger.kernel.org > > > > > >> More majordomo info at > > > > > >> http://vger.kernel.org/majordomo-info.html > > > > > >> > > > > > >> > > > > > > -- > > > > > > To unsubscribe from this list: send the line "unsubscribe > > > > > > ceph-devel" in the body of a message to > > > majordomo@vger.kernel.org > > > > > > More majordomo info at > > > http://vger.kernel.org/majordomo-info.html > > > > > > > > > > > > > > > > -- > > > > > Loïc Dachary, Artisan Logiciel Libre > > > > > > > > > > > > > > -- > > > > To unsubscribe from this list: send the line "unsubscribe > > > ceph-devel" > > > > in the body of a message to majordomo@vger.kernel.org More > > majordomo > > > > > > > info at http://vger.kernel.org/majordomo-info.html > > > > > > > > > > > > -- > > Matt Benjamin > > CohortFS, LLC. > > 315 West Huron Street, Suite 140A > > Ann Arbor, Michigan 48103 > > > > http://cohortfs.com > > > > tel. 734-761-4689 > > fax. 734-769-8938 > > cel. 734-216-5309 > > > > ________________________________ > > > > PLEASE NOTE: The information contained in this electronic mail > > message is intended only for the use of the designated recipient(s) > > named above. If the reader of this message is not the intended > > recipient, you are hereby notified that you have received this > > message in error and that any review, dissemination, distribution, > > or copying of this message is strictly prohibited. If you have > > received this communication in error, please notify the sender by > > telephone or e-mail (as shown above) immediately and destroy any and > > all copies of this message in your possession (whether hard copies > > or electronically > stored copies). > > > > N r y b X ?v ^ )?{.n + z ]z > > N r y b X ?v ^ )?{.n + z ]z {ay ?? ,j f h z  w j:+v w j m zZ+ > > ?j" ! i > >  ? & ~ &  +- ? w ? m  ^ b ^n r z  h &  G h ( ? ?j"  m > z > > ? f h ~ m > >  {.n + +% lzwm b ? r y? ?zX  ?} ?z &j:+v zZ+ +zf h ~ i z  w > > ? & )?f > > ________________________________ > > PLEASE NOTE: The information contained in this electronic mail message > is intended only for the use of the designated recipient(s) named > above. If the reader of this message is not the intended recipient, > you are hereby notified that you have received this message in error > and that any review, dissemination, distribution, or copying of this > message is strictly prohibited. If you have received this > communication in error, please notify the sender by telephone or > e-mail (as shown above) immediately and destroy any and all copies of > this message in your possession (whether hard copies or electronically stored copies). ________________________________ PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies). diff --git a/src/os/filestore/Makefile.am b/src/os/filestore/Makefile.am index de8ec07..77dbc4d 100644 --- a/src/os/filestore/Makefile.am +++ b/src/os/filestore/Makefile.am @@ -71,7 +71,7 @@ endif libceph_filestore_la_LDFLAGS = ${AM_LDFLAGS} -version-info 1:0:0 if LINUX -libceph_filestore_la_LDFLAGS += -export-symbols-regex '.*__ceph_plugin_.*' +#libceph_filestore_la_LDFLAGS += -export-symbols-regex '.*__ceph_plugin_.*' endif lib_LTLIBRARIES += libceph_filestore.la Thanks,