On Mon, Mar 18, 2019 at 10:06 AM Milind Changire <mchangir@redhat.com> wrote:
On Sun, Mar 17, 2019 at 4:40 PM Jeff Layton <jlayton@redhat.com> wrote:
On Sun, 2019-03-17 at 09:39 +0530, Milind Changire wrote:
> I could run cephfs-shell without 'make install'.
>
> After running vstart, cd to build directory and run:
> $ LD_LIBRARY_PATH=$PWD/lib python3.7 ../src/tools/cephfs/cephfs-shell --config ceph.conf
>
> The ceph.conf is generated when vstart is executed. I guess you'd know this part.
>
>
> On Sat, Mar 16, 2019 at 4:40 PM Jeff Layton <jlayton@redhat.com> wrote:
> > I need to do a bit of testing with cephfs and would like to avoid having
> > to use FUSE or write C calls into libcephfs. cephfs-shell seems perfect
> > for this, but I haven't quite figured out how to run it out of the build
> > directory.
> >
> > Is there a way to run cephfs-shell from a ceph build directory (i.e. w/o
> > make installing it)? Are there env vars that can be set to make it find
> > the right bits in the right places?
> >
> > Thanks,
> > --
> > Jeff Layton <jlayton@redhat.com>
> > _______________________________________________
> > ceph-fs mailing list -- ceph-fs@ceph.io
> > To unsubscribe send an email to ceph-fs-leave@ceph.io
>
>

Thanks Milind,

Hmm...that didn't work for me:

$ LD_LIBRARY_PATH=$PWD/lib python3.7 ../src/tools/cephfs/cephfs-shell --config ceph.conf
Traceback (most recent call last):
  File "../src/tools/cephfs/cephfs-shell", line 9, in <module>
    import cephfs as libcephfs
ModuleNotFoundError: No module named 'cephfs'

sorry about that ... I too tried a number of things
you probably want to run the following command under the build directory:
$ sudo ldconfig $PWD/lib

and then run the command I mentioned earlier.

Also, I had indeed done a sudo make install before I figured out the LD_LIBRARY_PATH kludge ... but now have found the following python shared objects even after a make uninstall. I wonder if these are the files that help in running the cephfs-shell:
$ find  /usr/local/lib64/python3.7/site-packages/ -name '*.so'
/usr/local/lib64/python3.7/site-packages/rados.cpython-37m-x86_64-linux-gnu.so
/usr/local/lib64/python3.7/site-packages/cephfs.cpython-37m-x86_64-linux-gnu.so
/usr/local/lib64/python3.7/site-packages/rgw.cpython-37m-x86_64-linux-gnu.so
/usr/local/lib64/python3.7/site-packages/rbd.cpython-37m-x86_64-linux-gnu.so

Also, the above shared objects are not owned by any RPM. So, looks like they never got cleaned up by the make uninstall.
$ rpm -q --whatprovides /usr/local/lib64/python3.7/site-packages/cephfs.cpython-37m-x86_64-linux-gnu.so
file /usr/local/lib64/python3.7/site-packages/cephfs.cpython-37m-x86_64-linux-gnu.so is not owned by any package

May be there are other libs left over by the make unsintall that are helping me run the cephfs-shell.

I'm guessing maybe you have a python3-cephfs module installed on your
machine?

No python3-cephfs on my system:
$ rpm -q python3-cephfs
package python3-cephfs is not installed

 
In any case I tried this and it got me a little closer:

$ PYTHONPATH=../src/pybind LD_LIBRARY_PATH=$PWD/lib python3.7 ../src/tools/cephfs/cephfs-shell --config ceph.conf
Traceback (most recent call last):
  File "../src/tools/cephfs/cephfs-shell", line 974, in <module>
    setup_cephfs(config_file)
  File "../src/tools/cephfs/cephfs-shell", line 59, in setup_cephfs
    cephfs = libcephfs.LibCephFS(conffile=config_file)
AttributeError: module 'cephfs' has no attribute 'LibCephFS'

...but I think it didn't import the module as expected. I'll keep
tinkering with it tomorrow.

Cheers,
--
Jeff Layton <jlayton@redhat.com>



I just noticed that the following lines are emitted by the vtstart script right at the end:

export PYTHONPATH=./pybind:/home/mchangir/work/ceph.git/src/pybind:/home/mchangir/work/ceph.git/build/lib/cython_modules/lib.3:
export LD_LIBRARY_PATH=/home/mchangir/work/ceph.git/build/lib
CEPH_DEV=1

Maybe, these will help ?

After I executed the above lines in my shell and moved out the libs under the /usr/local/lib64/python3.7/site-packages to a temporary directory, I could run cephfs-shell without any problems. I didn't need to prefix the command with LD_LIBRARY_PATH since that's exactly what the vstart script spewed out.