ceph-dashboard python warning with new pyo3 0.17 lib (debian12)
Hi, on debian12, ceph-dashboard is throwing a warning "Module 'dashboard' has failed dependency: PyO3 modules may only be initialized once per interpreter process" Seem to be related to pyo3 0.17 change https://github.com/PyO3/pyo3/blob/7bdc504252a2f972ba3490c44249b202a4ce6180/g... " Each #[pymodule] can now only be initialized once per process To make PyO3 modules sound in the presence of Python sub-interpreters, for now it has been necessary to explicitly disable the ability to initialize a #[pymodule] more than once in the same process. Attempting to do this will now raise an ImportError. "
Hello. Update to the latest versions of Ceph solves that. See https://docs.ceph.com/en/quincy/install/get-packages/ and https://download.ceph.com
Hello there, could you perhaps provide some more information on how (or where) this got fixed? It doesn't seem to be fixed yet on the latest Ceph Quincy and Reef versions, but maybe I'm mistaken. I've provided some more context regarding this below, in case that helps. On Ceph Quincy 17.2.6 I'm encountering the following error when trying to enable the dashboard (so, the same error that was posted above): root@ceph-01:~# ceph --version ceph version 17.2.6 (810db68029296377607028a6c6da1ec06f5a2b27) quincy (stable) root@ceph-01:~# ceph mgr module enable dashboard Error ENOENT: module 'dashboard' reports that it cannot run on the active manager daemon: PyO3 modules may only be initialized once per interpreter process (pass --force to force enablement) I was then able to find this Python traceback in the systemd journal: Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: 2023-09-04T18:39:51.438+0200 7fecdc91e000 -1 mgr[py] Traceback (most recent call last): Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/usr/share/ceph/mgr/dashboard/__init__.py", line 60, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from .module import Module, StandbyModule # noqa: F401 Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/usr/share/ceph/mgr/dashboard/module.py", line 30, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from .controllers import Router, json_error_page Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/usr/share/ceph/mgr/dashboard/controllers/__init__.py", line 1, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from ._api_router import APIRouter Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/usr/share/ceph/mgr/dashboard/controllers/_api_router.py", line 1, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from ._router import Router Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/usr/share/ceph/mgr/dashboard/controllers/_router.py", line 7, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from ._base_controller import BaseController Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/usr/share/ceph/mgr/dashboard/controllers/_base_controller.py", line 11, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from ..services.auth import AuthManager, JwtManager Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/usr/share/ceph/mgr/dashboard/services/auth.py", line 12, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: import jwt Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/lib/python3/dist-packages/jwt/__init__.py", line 1, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from .api_jwk import PyJWK, PyJWKSet Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/lib/python3/dist-packages/jwt/api_jwk.py", line 6, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from .algorithms import get_default_algorithms Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/lib/python3/dist-packages/jwt/algorithms.py", line 6, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from .utils import ( Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/lib/python3/dist-packages/jwt/utils.py", line 7, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from cryptography.hazmat.primitives.asymmetric.ec import EllipticCurve Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/lib/python3/dist-packages/cryptography/hazmat/primitives/asymmetric/ec.py", line 11, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from cryptography.hazmat._oid import ObjectIdentifier Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/lib/python3/dist-packages/cryptography/hazmat/_oid.py", line 7, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from cryptography.hazmat.bindings._rust import ( Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: ImportError: PyO3 modules may only be initialized once per interpreter process Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: 2023-09-04T18:39:51.438+0200 7fecdc91e000 -1 mgr[py] Class not found in module 'dashboard' Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: 2023-09-04T18:39:51.438+0200 7fecdc91e000 -1 mgr[py] Error loading module 'dashboard': (2) No such file or directory Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: 2023-09-04T18:39:51.470+0200 7fecdc91e000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: 2023-09-04T18:39:51.502+0200 7fecdc91e000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: 2023-09-04T18:39:51.502+0200 7fecdc91e000 -1 log_channel(cluster) log [ERR] : Failed to load ceph-mgr modules: dashboard As the traceback above reveals, the dashboard uses `PyJWT`, which in turn uses `cryptography`, and `cryptography` uses `PyO3`. That led me to an issue[0] regarding this on `cryptography`'s side; the Ceph Dashboard is apparently not the only thing that's affected by this. As it turns out, the maintainer of the Ceph AUR package has also recently stumbled across this while re-packaging Ceph Reef 18.2.x and has listed many more details regarding this issue[1]. Hope this helps! [0]: https://github.com/pyca/cryptography/issues/9016 [1]: https://github.com/bazaah/aur-ceph/issues/20
On 9/5/23 16:53, Max Carrara wrote:
Hello there,
could you perhaps provide some more information on how (or where) this got fixed? It doesn't seem to be fixed yet on the latest Ceph Quincy and Reef versions, but maybe I'm mistaken. I've provided some more context regarding this below, in case that helps.
On Ceph Quincy 17.2.6 I'm encountering the following error when trying to enable the dashboard (so, the same error that was posted above):
root@ceph-01:~# ceph --version ceph version 17.2.6 (810db68029296377607028a6c6da1ec06f5a2b27) quincy (stable)
root@ceph-01:~# ceph mgr module enable dashboard Error ENOENT: module 'dashboard' reports that it cannot run on the active manager daemon: PyO3 modules may only be initialized once per interpreter process (pass --force to force enablement)
I was then able to find this Python traceback in the systemd journal:
Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: 2023-09-04T18:39:51.438+0200 7fecdc91e000 -1 mgr[py] Traceback (most recent call last): Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/usr/share/ceph/mgr/dashboard/__init__.py", line 60, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from .module import Module, StandbyModule # noqa: F401 Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/usr/share/ceph/mgr/dashboard/module.py", line 30, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from .controllers import Router, json_error_page Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/usr/share/ceph/mgr/dashboard/controllers/__init__.py", line 1, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from ._api_router import APIRouter Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/usr/share/ceph/mgr/dashboard/controllers/_api_router.py", line 1, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from ._router import Router Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/usr/share/ceph/mgr/dashboard/controllers/_router.py", line 7, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from ._base_controller import BaseController Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/usr/share/ceph/mgr/dashboard/controllers/_base_controller.py", line 11, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from ..services.auth import AuthManager, JwtManager Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/usr/share/ceph/mgr/dashboard/services/auth.py", line 12, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: import jwt Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/lib/python3/dist-packages/jwt/__init__.py", line 1, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from .api_jwk import PyJWK, PyJWKSet Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/lib/python3/dist-packages/jwt/api_jwk.py", line 6, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from .algorithms import get_default_algorithms Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/lib/python3/dist-packages/jwt/algorithms.py", line 6, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from .utils import ( Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/lib/python3/dist-packages/jwt/utils.py", line 7, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from cryptography.hazmat.primitives.asymmetric.ec import EllipticCurve Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/lib/python3/dist-packages/cryptography/hazmat/primitives/asymmetric/ec.py", line 11, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from cryptography.hazmat._oid import ObjectIdentifier Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: File "/lib/python3/dist-packages/cryptography/hazmat/_oid.py", line 7, in <module> Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: from cryptography.hazmat.bindings._rust import ( Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: ImportError: PyO3 modules may only be initialized once per interpreter process Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: 2023-09-04T18:39:51.438+0200 7fecdc91e000 -1 mgr[py] Class not found in module 'dashboard' Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: 2023-09-04T18:39:51.438+0200 7fecdc91e000 -1 mgr[py] Error loading module 'dashboard': (2) No such file or directory Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: 2023-09-04T18:39:51.470+0200 7fecdc91e000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: 2023-09-04T18:39:51.502+0200 7fecdc91e000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member Sep 04 18:39:51 ceph-01 ceph-mgr[15669]: 2023-09-04T18:39:51.502+0200 7fecdc91e000 -1 log_channel(cluster) log [ERR] : Failed to load ceph-mgr modules: dashboard
As the traceback above reveals, the dashboard uses `PyJWT`, which in turn uses `cryptography`, and `cryptography` uses `PyO3`.
That led me to an issue[0] regarding this on `cryptography`'s side; the Ceph Dashboard is apparently not the only thing that's affected by this.
As it turns out, the maintainer of the Ceph AUR package has also recently stumbled across this while re-packaging Ceph Reef 18.2.x and has listed many more details regarding this issue[1].
Hope this helps!
[0]: https://github.com/pyca/cryptography/issues/9016 [1]: https://github.com/bazaah/aur-ceph/issues/20 _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io
Is there perhaps any update on this? Thanks!
participants (3)
-
David Fojtík
-
DERUMIER, Alexandre
-
Max Carrara