>Except for your mds, mrg and radosgw, your osd
daemons are bound to
>the hardware / disks they are running on. It is not like if
> osd.121 goes down, you can start it on some random node.
Why not?
The data stays on the old node not? If you did automate destroy/create
of a new osd, that would also mean that the cluster has to replicate
data to this 'new' osd.121, if there is even storage available on that
node. And if there is indeed storage available and not used at that
node, why not use it in the first place for an osd.122?