Hi, I want to write and read from an rbd image with librbd (librbdpy or go-ceph). I got some questions: 1. How can I do a random write/read with it? 2. How can I delete the written data with it? 3. If I want to write with a for example 4K block size should I break my bytes array to 4K arrays by my self or librbd will do it on its own? 4. Does rbd_cache included in it? If yes how can I disable it? Thanks.
On Thursday, December 24, 2020 8:25:15 AM EST Seena Fallah wrote:
Hi,
I want to write and read from an rbd image with librbd (librbdpy or go-ceph). I got some questions:
I'll try to touch on some of the below for go-ceph specifically. However, the principles should generally apply to librdb (C) or python as all three libraries are ultimately backed by the same stuff. Our naming should largely match what you'd find in librbd but occasionally we need to change things a little bit to match Go naming conventions and common interfaces [0].
1. How can I do a random write/read with it?
I would use the WriteAt [1] and ReadAt [2] calls on the Image type.
2. How can I delete the written data with it?
I'm not entirely clear on this question. You can delete an entire image. Or you could also zero out a range of data within the image either using something like Discard [3] or WriteSame [4].
3. If I want to write with a for example 4K block size should I break my bytes array to 4K arrays by my self or librbd will do it on its own?
If I understand the question correctly, you do not need to read and write with the same underlying stripe/block size as of the image itself. If you wanted to write a 16 bytes to an image you can just pass it a 16 byte long slice (to use the Go term). You may find certain buffer sizes perform better than others, but the library doesn't require it, IIUC.
4. Does rbd_cache included in it? If yes how can I disable it?
Unfortunately, I'm afraid I can't help with this one without more info as I'm not sure what it refers to. Perhaps someone with deeper experience specifically in rbd may know. I can say that go-ceph can load any ceph conf file, passing it to the underlying ceph libraries, so if its something you can disable for the rbd command line tool you ought to be able to do so for go-ceph [5].
Thanks.
[0] - We try to document what ceph API functions are being wrapped but we're not perfect and not all older code has had these "Implements" sections added to the inline docs. [1] - https://pkg.go.dev/github.com/ceph/go-ceph@v0.7.0/rbd#Image.WriteAt [2] - https://pkg.go.dev/github.com/ceph/go-ceph@v0.7.0/rbd#Image.ReadAt [3] - https://pkg.go.dev/github.com/ceph/go-ceph@v0.7.0/rbd#Image.Discard [4] - https://pkg.go.dev/github.com/ceph/go-ceph@v0.7.0/rbd#Image.WriteSame [5] - For rbd use the set up functions in the rados module to get an IOContext. https://pkg.go.dev/github.com/ceph/go-ceph@v0.7.0/ rados#Conn.ReadDefaultConfigFile - Please file issues with go-ceph if you try it and something does not work.
If I understand the question correctly, you do not need to read and write with
Thanks a lot! the same underlying stripe/block size as of the image itself. If you wanted to write a 16 bytes to an image you can just pass it a 16 byte long slice (to use the Go term). Do you mean that if I pass the 10MiB bytes array to WriteAt func with default image configs it will strip it into 2 4MiB requests? for example, if I mount an Image to Linux I can specify the block size and each IO will be 4K for example and if I'm not wrong each op in Ceph will be 4K too. Now how can I simulate that behavior here too? I'm also facing an issue with go-ceph that with a 5-second interval I'm discarding and writing data to an image. After a while, for example, 30 sec the write will be stuck and it doesn't throw any error! How can I check what is wrong with it? cluster health is OK. On Thu, Dec 24, 2020 at 7:05 PM John Mulligan <phlogistonjohn@asynchrono.us> wrote:
On Thursday, December 24, 2020 8:25:15 AM EST Seena Fallah wrote:
Hi,
I want to write and read from an rbd image with librbd (librbdpy or go-ceph). I got some questions:
I'll try to touch on some of the below for go-ceph specifically. However, the principles should generally apply to librdb (C) or python as all three libraries are ultimately backed by the same stuff. Our naming should largely match what you'd find in librbd but occasionally we need to change things a little bit to match Go naming conventions and common interfaces [0].
1. How can I do a random write/read with it?
I would use the WriteAt [1] and ReadAt [2] calls on the Image type.
2. How can I delete the written data with it?
I'm not entirely clear on this question. You can delete an entire image. Or you could also zero out a range of data within the image either using something like Discard [3] or WriteSame [4].
3. If I want to write with a for example 4K block size should I break my bytes array to 4K arrays by my self or librbd will do it on its own?
If I understand the question correctly, you do not need to read and write with the same underlying stripe/block size as of the image itself. If you wanted to write a 16 bytes to an image you can just pass it a 16 byte long slice (to use the Go term).
You may find certain buffer sizes perform better than others, but the library doesn't require it, IIUC.
4. Does rbd_cache included in it? If yes how can I disable it?
Unfortunately, I'm afraid I can't help with this one without more info as I'm not sure what it refers to. Perhaps someone with deeper experience specifically in rbd may know.
I can say that go-ceph can load any ceph conf file, passing it to the underlying ceph libraries, so if its something you can disable for the rbd command line tool you ought to be able to do so for go-ceph [5].
Thanks.
[0] - We try to document what ceph API functions are being wrapped but we're not perfect and not all older code has had these "Implements" sections added to the inline docs. [1] - https://pkg.go.dev/github.com/ceph/go-ceph@v0.7.0/rbd#Image.WriteAt [2] - https://pkg.go.dev/github.com/ceph/go-ceph@v0.7.0/rbd#Image.ReadAt [3] - https://pkg.go.dev/github.com/ceph/go-ceph@v0.7.0/rbd#Image.Discard [4] - https://pkg.go.dev/github.com/ceph/go-ceph@v0.7.0/rbd#Image.WriteSame [5] - For rbd use the set up functions in the rados module to get an IOContext. https://pkg.go.dev/github.com/ceph/go-ceph@v0.7.0/ rados#Conn.ReadDefaultConfigFile - Please file issues with go-ceph if you try it and something does not work.
participants (2)
-
John Mulligan
-
Seena Fallah