Hi Vitaliy,
Awesome what you did. I like the minimalistic approach focused on
performance very much. A long way you've passed implementing the
whole stack from client to storage.
Could you please describe peering process and object recovery in more
details? You write in the README that there is no PGLog in your
storage. After a brief glance into osd_ops.h and blockstore_impl.cpp I
see there is a 'version' field. I can assume that every object is
marked with a version and recovery happens taking the whole up-to-date
object, right?
--
Roman
On 2020-09-23 00:44, vitalif(a)yourcmc.ru wrote:
Hi!
After almost a year of development in my spare time I present my own
software-defined block storage system: Vitastor -
https://vitastor.io
I designed it similar to Ceph in many ways, it also has Pools, PGs,
OSDs, different coding schemes, rebalancing and so on. However it's
much simpler and much faster. In a test cluster with SATA SSDs it
achieved Q1T1 latency of 0.14ms which is especially great compared to
Ceph RBD's 1ms for writes and 0.57ms for reads. In an "iops
saturation" parallel load benchmark it reached 895k read / 162k write
iops, compared to Ceph's 480k / 100k on the same hardware, but the
most interesting part was CPU usage: Ceph OSDs were using 40 CPU cores
out of 64 on each node and Vitastor was only using 4.
Of course it's an early pre-release which means that, for example, it
lacks snapshot support and other useful features. However the base is
finished - it works and runs QEMU VMs. I like the design and I plan to
develop it further.
There are more details in the README file which currently opens from
the domain
https://vitastor.io
Sorry if it was a bit off-topic, I just thought it could be
interesting for you :)