On Sunday, September 27, 2020 12:41:32 PM EDT vitalif@yourcmc.ru wrote:
> I suspect it's impossible to notice cgo overhead with Ceph :))
>

I would be a bit somewhat surprised to hear if it really was cgo itself too.
But I have some additional thoughts on that below.



> > Hi John,
> >
> > Thanks for the quick response. I am aware of go-ceph. My team has used it
> > successfully for several control-plane services. However, the performance
> > penalty imposed by the Cgo runtime makes go-ceph not a good fit for the
> > data-plane services we'd like to build in the future.

Sanford, do you have any benchmarks or any pointers to what methods you tested
and were too slow for your use cases? It may be that the go-ceph
implementation performs too many data copies. Some of the methods that work
with strings rather than byte slices as buffers need to do copies between C/C+
+ memory to Go memory and we may be incurring too large of a performance
penalty for that.

Even if you don't use go-ceph for your future projects we'd like to know where
we can improve. Right now, I've been more focused on expanding our coverage of
the functionality in the lib{rados,rbd,cephfs} wrapper packages, but being
efficient is important too.

> > The idea for the
> > library I want to create/start is to have serialization and
> > deserialization, of both the RADOS client protocol and of the messenger
> > protocol wrapping it, done natively in Go. Hence, it would be a huge help
> > to get some pointers to good sources of information about the RADOS
> > client protocol.

I'll certainly want to keep tabs on any effort here if you go forward with it.
For my use cases using wrappers gets us to our goals faster but there's
certainly some trade offs we make when we use cgo. One big one for me is the
lack of deeper integration with goroutines and the difficulty using common Go
patterns like deadlines on context.Context.


Thanks!