python3-rados issue when configuring cluster handle

16 Apr 2020

      Hello! I am seeing a TypeError when configuring a cluster handle, and it seems like it is *probably* some sort of versioning/build related issue, but I wanted to get some clarity here. Also, if it is a bug, I am not exactly sure how to register it, so some pointers there would be great.

The description of the issue is that the rados.py module (<path-to-pythonlib>/python3.6/site-packages/rados.py) has code of the following nature:
        ...
        if clustername is not None and not isinstance(clustername, str):
            raise TypeError('clustername must be a string or None')
        ...
        ret = run_in_thread(
            self.librados.rados_create2,
            (
                byref(self.cluster),
                c_char_p(clustername),
                c_char_p(name),
                c_uint64(flags)
            )
        )

but calling `c_char_p(<str>)` throws a TypeError ("bytes or integer address expected instead of str instance"). This leads me to believe that at some point, or for some versions, c_char_p took a string and now for my environment it doesn't. I went through the rados.py file and byte-encoded the strings (using "string".encode('utf-8')), and the TypeError is no longer thrown, but I haven't yet tested much further in order to know if I've shot myself in the foot. I tried looking for the source, but couldn't quite figure out where this code is generated from, or kept. The closest I seem to be to finding the source is this file: https://github.com/ceph/ceph/blob/master/src/pybind/rados/rados.pyx

I would appreciate some help/insight/guidance as to a proper fix, if this bug was filed before (how could it not? but also I couldn't find when searching the issue tracker), and where to report if it is unreported.

Thanks!

------------------------------

Verbose, relevant information below:

    A minimal script to reproduce the errors:
       https://gist.github.com/drin/461942b0a361053203607cb5eb17cac4

    The error:
        TypeError: bytes or integer address expected instead of str instance

    The environment:
       cat /etc/os-release
          NAME="Ubuntu"
          VERSION="18.04.1 LTS (Bionic Beaver)"

       rados package installed:
          python3-rados_12.2.12-0ubuntu0.18.04.5_amd64.deb

       A 2-node cluster (1 client, 1 osd), setup in the following way on cloudlab:
          https://github.com/uccross/skyhookdm-ceph/wiki/Ceph-SkyhookDM-cluster-setup#...

    The python interpreter version and dependencies:
       Python 3.6.9

        atomicwrites       1.3.0
        attrs              19.3.0
        awscli             1.18.39
        botocore           1.15.39
        colorama           0.4.3
        cython             0.29.16
        docutils           0.15.2
        flatbuffers        1.11
        h5py               2.10.0
        importlib-metadata 1.6.0
        jmespath           0.9.5
        more-itertools     8.2.0
        numpy              1.18.2
        owlready2          0.23
        pandas             1.0.3
        pluggy             0.13.1
        py                 1.8.1
        pyarrow            0.15.1
        pyasn1             0.4.8
        pytest             3.10.1
        python-dateutil    2.8.1
        pytz               2019.3
        pyyaml             5.3.1
        rsa                3.4.2
        s3transfer         0.3.3
        scipy              1.4.1
        six                1.14.0
        urllib3            1.25.9
        zipp               3.1.0

akmontan＠ucsc.edu

Brad Hubbard

akmontan＠ucsc.edu

tags

participants (2)