28 Feb
2020
28 Feb
'20
12:39 p.m.
Hi,
I'm the first to acknowledge that I do not know enough of python.
But still I can get by most of the times.
However during the tests of my Ceph port one of the tests complains:
==============
orchestrator/_interface.py:701: ImportError
------------------------------ Captured log call -------------------------------
ERROR orchestrator._interface:_interface.py:391 _Promise failed
Traceback (most recent call last):
File "/home/jenkins/workspace/ceph-master/src/pybind/mgr/cephadm/module.py",
line 334, in do_work
res = self._on_complete_(*args, **kwargs)
File "/home/jenkins/workspace/ceph-master/src/pybind/mgr/cephadm/module.py",
line 398, in call_self
return f(self, *inner_args)
File "/home/jenkins/workspace/ceph-master/src/pybind/mgr/cephadm/module.py",
line 2352, in _create_grafana
return self._create_daemon('grafana', daemon_id, host)
File "/home/jenkins/workspace/ceph-master/src/pybind/mgr/cephadm/module.py",
line 1874, in _create_daemon
j = self._generate_grafana_config()
File "/home/jenkins/workspace/ceph-master/src/pybind/mgr/cephadm/module.py",
line 2288, in _generate_grafana_config
cert, pkey = create_self_signed_cert('Ceph', 'cephadm')
File "/home/jenkins/workspace/ceph-master/src/pybind/mgr/mgr_util.py", line
134, in create_self_signed_cert
from OpenSSL import crypto
File
"/home/jenkins/workspace/ceph-master/src/pybind/mgr/.tox/py3/lib/python3.7/site-packages/OpenSSL/__init__.py",
line 8, in <module>
from OpenSSL import crypto, SSL
File
"/home/jenkins/workspace/ceph-master/src/pybind/mgr/.tox/py3/lib/python3.7/site-packages/OpenSSL/crypto.py",
line 15, in <module>
from OpenSSL._util import (
File
"/home/jenkins/workspace/ceph-master/src/pybind/mgr/.tox/py3/lib/python3.7/site-packages/OpenSSL/_util.py",
line 6, in <module>
from cryptography.hazmat.bindings.openssl.binding import Binding
File
"/home/jenkins/workspace/ceph-master/src/pybind/mgr/.tox/py3/lib/python3.7/site-packages/cryptography/hazmat/bindings/openssl/binding.py",
line 15, in <module>
from cryptography.hazmat.bindings._openssl import ffi, lib
ImportError:
/home/jenkins/workspace/ceph-master/src/pybind/mgr/.tox/py3/lib/python3.7/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so:
Undefined symbol "SSLv3_client_method"
==============
This is due to the fact that on FreeBSD openSSL has its SSLv3 code disabled.
Now this is fixable on an individual basis, by recompiling the openSSL
port with SSLv3 enabled.
But for a generic port that is not really an option. The user than has
to jump thru loops to build
its own openSSL, and even then he/she needs to keep up with security
isssues. One should not want this.
The problem stems from virtualenv/tox fetching from public sources,
instead of using the ports system.
This can be overruled by: --system-site-packages.
I know that I'll need to load all packages before running
virtualenv/tox, but that is "just" a matter of
collecting the list.
but I'm wondering if this is a feasable solution?
--WjW
28 Feb
28 Feb
3:59 p.m.
New subject: Missing symbol under tox in openssl stuff
On Fri, Feb 28, 2020 at 5:39 PM Willem Jan Withagen <wjw(a)digiware.nl> wrote:
Hi,
I'm the first to acknowledge that I do not know enough of python.
But still I can get by most of the times.
However during the tests of my Ceph port one of the tests complains:
==============
orchestrator/_interface.py:701: ImportError
------------------------------ Captured log call -------------------------------
ERROR orchestrator._interface:_interface.py:391 _Promise failed
Traceback (most recent call last):
File "/home/jenkins/workspace/ceph-master/src/pybind/mgr/cephadm/module.py",
line 334, in do_work
res = self._on_complete_(*args, **kwargs)
File "/home/jenkins/workspace/ceph-master/src/pybind/mgr/cephadm/module.py",
line 398, in call_self
return f(self, *inner_args)
File "/home/jenkins/workspace/ceph-master/src/pybind/mgr/cephadm/module.py",
line 2352, in _create_grafana
return self._create_daemon('grafana', daemon_id, host)
File "/home/jenkins/workspace/ceph-master/src/pybind/mgr/cephadm/module.py",
line 1874, in _create_daemon
j = self._generate_grafana_config()
File "/home/jenkins/workspace/ceph-master/src/pybind/mgr/cephadm/module.py",
line 2288, in _generate_grafana_config
cert, pkey = create_self_signed_cert('Ceph', 'cephadm')
File "/home/jenkins/workspace/ceph-master/src/pybind/mgr/mgr_util.py", line
134, in create_self_signed_cert
from OpenSSL import crypto
File
"/home/jenkins/workspace/ceph-master/src/pybind/mgr/.tox/py3/lib/python3.7/site-packages/OpenSSL/__init__.py",
line 8, in <module>
from OpenSSL import crypto, SSL
File
"/home/jenkins/workspace/ceph-master/src/pybind/mgr/.tox/py3/lib/python3.7/site-packages/OpenSSL/crypto.py",
line 15, in <module>
from OpenSSL._util import (
File
"/home/jenkins/workspace/ceph-master/src/pybind/mgr/.tox/py3/lib/python3.7/site-packages/OpenSSL/_util.py",
line 6, in <module>
from cryptography.hazmat.bindings.openssl.binding import Binding
File
"/home/jenkins/workspace/ceph-master/src/pybind/mgr/.tox/py3/lib/python3.7/site-packages/cryptography/hazmat/bindings/openssl/binding.py",
line 15, in <module>
from cryptography.hazmat.bindings._openssl import ffi, lib
ImportError:
/home/jenkins/workspace/ceph-master/src/pybind/mgr/.tox/py3/lib/python3.7/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so:
Undefined symbol "SSLv3_client_method"
==============
This is due to the fact that on FreeBSD openSSL has its SSLv3 code disabled.
if that's the case, OPENSSL_NO_SSL3_METHOD should be defined, and
hence cryptography should have this symbol defined, see
https://github.com/pyca/cryptography/blob/7247665f76cf849fb5b3020a28cfc86c4…
.
could you check /usr/local/include/openssl/opensslfeatures.h on your build host?
Now this is fixable on an individual basis, by
recompiling the openSSL port with SSLv3 enabled.
But for a generic port that is not really an option. The user than has to jump thru loops
to build
its own openSSL, and even then he/she needs to keep up with security isssues. One should
not want this.
The problem stems from virtualenv/tox fetching from public sources, instead of using the
ports system.
This can be overruled by: --system-site-packages.
I know that I'll need to load all packages before running virtualenv/tox, but that is
"just" a matter of
collecting the list.
but I'm wondering if this is a feasable solution?
--WjW
_______________________________________________
Dev mailing list -- dev(a)ceph.io
To unsubscribe send an email to dev-leave(a)ceph.io
--
Regards
Kefu Chai
5:01 p.m.
New subject: Missing symbol under tox in openssl stuff
On 28-2-2020 13:59, kefu chai wrote:
On Fri, Feb 28, 2020 at 5:39 PM Willem Jan Withagen
<wjw(a)digiware.nl> wrote:
That file does (no longer) exist in my /usr/local...
I checked by looking in the ports makefile, which registers the
defaults. And you can check this when running make config, it'll also
tell you that SSLv3 is off.
What I do find in /usr/local/include/openssl is:
/usr/local/include/openssl/ssl.h:DEPRECATEDIN_1_1_0(__owur const
SSL_METHOD *SSLv3_method(void)) /* SSLv3 */
/usr/local/include/openssl/ssl.h:DEPRECATEDIN_1_1_0(__owur const
SSL_METHOD *SSLv3_server_method(void))
/usr/local/include/openssl/ssl.h:DEPRECATEDIN_1_1_0(__owur const
SSL_METHOD *SSLv3_client_method(void))
And the openssl in base is 1.1.0, and in ports is 1.1.1 so in both it
should be deprecated....
But in the ports Makefile it is not default on:
OPTIONS_DEFAULT=ASM ASYNC CT GOST DES EC MAN3 MD4 NEXTPROTONEG RC2 RC4 \
RMD160 SCTP SHARED SSE2 THREADS TLS1 TLS1_1 TLS1_2
Now this issue comes in a few flavours:
Currently this only breaks in master, when with run-tox-mgr.
This I can fix on by jenkins builder by manually installing a libcrypto
that does take SSLv3.
As long as the ports I make do not include cephadm, things will be fine.
I still have not finished all the preparations for the ceph15 port.
Ceph14-14.2.7 has just been submitted.
So that will oke as long as nothing is being backported.
Once that happens those versions will also suffer.
Question is what happens when I release a package with cephadm in it
which requires py-cryptography? As long as the FreeBSD port of that also
does not need SSLv3 we're oke. Which will be since there is no tox involved.
So for this it is only a problem when tox does not install the FreeBSD
py-cryptography port but tries to fix it by itself.
Hence my question if it would be sensible to get virtualenv/tox/pip/....
to prefer FreeBSD ports first. Or maybe even get
src/tools/setup-virtualenv.sh
or src/script/run_tox.sh
to install if first before tox installs even more things (using py3??).
Would that make sense?
Now for something totally different with openssl versions:
I have a similar problem of incompatible crypto I think when building my
bhyve/rbd-device plugin where as things can get incompatible due to:
src/common/ceph_crypto.h:
namespace ceph::crypto::ssl {
# if OPENSSL_VERSION_NUMBER < 0x10100000L
And that if I try to run my plugin with a librados which is compiled on
a platform with openssl < 0x10100000L it will crash in HMAC routines.
Took me a while to figure that out. But that is more a development issue
and I'm running way too many different combinations of release of Ceph
and FreeBSD.
Perhaps it is possible to glue a const in like
namespace ceph::crypto::ssl {
...........
public:
const int rados_uses_ssl_version = OPENSSL_VERSION_NUMBER;
Such that it is possible to ERR-out with a decent errormessage if not
all versions match???
But then this needs to be mapped to C-code also, to be workable.
--WjW
Hi,
I'm the first to acknowledge that I do not know enough of python.
But still I can get by most of the times.
However during the tests of my Ceph port one of the tests complains:
==============
orchestrator/_interface.py:701: ImportError
------------------------------ Captured log call -------------------------------
ERROR orchestrator._interface:_interface.py:391 _Promise failed
Traceback (most recent call last):
File "/home/jenkins/workspace/ceph-master/src/pybind/mgr/cephadm/module.py",
line 334, in do_work
res = self._on_complete_(*args, **kwargs)
File "/home/jenkins/workspace/ceph-master/src/pybind/mgr/cephadm/module.py",
line 398, in call_self
return f(self, *inner_args)
File "/home/jenkins/workspace/ceph-master/src/pybind/mgr/cephadm/module.py",
line 2352, in _create_grafana
return self._create_daemon('grafana', daemon_id, host)
File "/home/jenkins/workspace/ceph-master/src/pybind/mgr/cephadm/module.py",
line 1874, in _create_daemon
j = self._generate_grafana_config()
File "/home/jenkins/workspace/ceph-master/src/pybind/mgr/cephadm/module.py",
line 2288, in _generate_grafana_config
cert, pkey = create_self_signed_cert('Ceph', 'cephadm')
File "/home/jenkins/workspace/ceph-master/src/pybind/mgr/mgr_util.py", line
134, in create_self_signed_cert
from OpenSSL import crypto
File
"/home/jenkins/workspace/ceph-master/src/pybind/mgr/.tox/py3/lib/python3.7/site-packages/OpenSSL/__init__.py",
line 8, in <module>
from OpenSSL import crypto, SSL
File
"/home/jenkins/workspace/ceph-master/src/pybind/mgr/.tox/py3/lib/python3.7/site-packages/OpenSSL/crypto.py",
line 15, in <module>
from OpenSSL._util import (
File
"/home/jenkins/workspace/ceph-master/src/pybind/mgr/.tox/py3/lib/python3.7/site-packages/OpenSSL/_util.py",
line 6, in <module>
from cryptography.hazmat.bindings.openssl.binding import Binding
File
"/home/jenkins/workspace/ceph-master/src/pybind/mgr/.tox/py3/lib/python3.7/site-packages/cryptography/hazmat/bindings/openssl/binding.py",
line 15, in <module>
from cryptography.hazmat.bindings._openssl import ffi, lib
ImportError:
/home/jenkins/workspace/ceph-master/src/pybind/mgr/.tox/py3/lib/python3.7/site-packages/cryptography/hazmat/bindings/_openssl.abi3.so:
Undefined symbol "SSLv3_client_method"
==============
This is due to the fact that on FreeBSD openSSL has its SSLv3 code disabled.
if that's the case, OPENSSL_NO_SSL3_METHOD should be defined, and
hence cryptography should have this symbol defined, see
https://github.com/pyca/cryptography/blob/7247665f76cf849fb5b3020a28cfc86c4…
.
could you check /usr/local/include/openssl/opensslfeatures.h on your build host?
1542
days inactive
1542
days old
2 comments
2 participants
participants (2)
-
kefu chai -
Willem Jan Withagen