Additionally, to make this comparison fairer, we should also take into account not only the code we currently have, but the quality and features we could deliver by relying on mature yet-not-packaged dependencies (I'm thinking on typing-extensions, pydantic, websockets or fastapi among others). Bringing fresh dependencies into the project right now involves ensuring that they're available for a combination of ~5-6 distro-release pairs (Ubuntu, CentOS, OpenSUSE, FreeBSD ports) and that's often a showstopper. -- Ernesto On Mon, Mar 8, 2021 at 1:26 PM Alfonso Martinez Hidalgo <almartin@redhat.com> wrote:
Hi all,
In the last weeks the mgr dashboard module has suffered several bugs related to python package versions:
1) mgr/dashboard: dashboard hangs when accessing it https://tracker.ceph.com/issues/48973 cheroot 8.5.1 was causing a hang: needed to upgrade to cheroot distro version 8.5.2
2) mgr/dashboard: ERROR: test_a_set_login_credentials (tasks.mgr.dashboard.test_auth.AuthTest) https://tracker.ceph.com/issues/49574
Adapt code to PyJWT version >= 2.0.0 (our tests were relying on version==1.6.4)
3) No possibility to use (in nautilus branch) features from "six" package version like "ensure_text" (available in version >=1.12) due to the fact that distros like ubuntu bionic seem to be stuck on version 1.11
Taking into consideration all the above and the fact that version pinning/constraint can provide stability & reliability, e.g.: https://github.com/psf/requests/blob/master/setup.py#L44 https://github.com/tiangolo/fastapi/blob/master/pyproject.toml#L34
PROPOSAL: Including a virtualenv (or equivalent) in mgr(dashboard) packag(es) (created during build process) with some pinned dependencies so we are no longer dependent on system packages for the specified packages.
We're already doing it for Dashboard Frontend (Angular Single Page App) npm dependencies:
https://github.com/ceph/ceph/blob/master/src/pybind/mgr/dashboard/frontend/p...
PROS: * Stability: no longer exposed to potential bugs coming from side-effects of third-party decisions, like distro maintainers package upgrade policy.
* Reliability: code tested against well-known package versions; avoid the problem of having disparity of package versions in several distros & distro releases.
* Flexibility: possibility of using packages that right now are not available on all supported distros (or required version not available).
CONS: * Package build size increase: we should measure the new size and maybe considering an incremental approach: as a first step, pinning only those packages that are "problematic" (have high version disparity across distros/releases, those not available on supported distros, etc).
* Taking care of CVE-related security updates for pinned versions: 1 possibility is tracking the security updates with tools like this:
https://docs.github.com/en/github/managing-security-vulnerabilities/configur...
Let me know any other considerations that we should take into account.
Regards, --
Alfonso Martínez
Senior Software Engineer, Ceph Storage
Red Hat