All the mira nodes are currently locked too. A few with the word "scheduled" but also 31 that look to be dead suite runs from Venky and Thomas — not sure if those were manually locked or if the stale-nodes cleanup script needs to be run against mira separately as well? -Greg On Thu, Feb 6, 2020 at 6:31 AM David Galloway <dgallowa@redhat.com> wrote:
I think Yuri usually runs a script at least once a day that cleans up stale nodes. He's on PTO so I just ran it and it cleaned up a few dozen systems.
On 2/6/20 2:38 AM, Brad Hubbard wrote:
$ teuthology-lock --summary|grep scheduled_|grep smithi smithi 1 1 scheduled_kchai@teuthology smithi 1 0 scheduled_rishabh@teuthology smithi 2 2 scheduled_yuvalif@teuthology smithi 2 2 scheduled_akupczyk@teuthology smithi 11 11 scheduled_bhubbard@teuthology smithi 12 11 scheduled_sage@teuthology smithi 26 26 scheduled_jcollin@teuthology smithi 35 35 scheduled_nojha@teuthology smithi 36 35 scheduled_teuthology@teuthology smithi 38 37 scheduled_pdonnell@teuthology
THis doesn't include the machines locked by individuals.
Kefu does not appear to have any runs at the moment and neither does Jos. I have one run going but all of my jobs are in 'waiting' state. Sage, Neha, and Patrick all have jobs running but not enough to explain the number of machines locked IMHO. It seems to me we may have some sort of leak happening at the moment which is greatly lowering the number of available machines. Kefu and I both looked at this and agree this does not look right.
_______________________________________________ Sepia mailing list -- sepia@ceph.io To unsubscribe send an email to sepia-leave@ceph.io