Hi,
I'd like to gain a better understanding about what operations emit which
of these performance counters, in particular when is 'op_rw' incremented
instead of 'op_r' + 'op_w'?
I've done a little bit of investigation (v12.2.13) , running various
workoads and operations against an RBD volume (in a cluster with no
other client activity):
- Most RBD 'operations' (create, rm, features disable/enable, map,
unmap) emit 'op_rw' and often 'op_w' too
- Program reads and writes against a mounted RBD volume *only* emit
'op_r' and 'op_w' (never 'op_rw'), regardless of whether they are
'read
+ modify' of existing file data (or whether the writes are buffered,
direct or sync)
Is that correct? Or have I missed a program driven workload that will
produce 'op_rw'? [1]
In our production clusters I'm seeing similar numbers of 'op_w' and
'op_rw' (for a given OSD), which would suggest a lot of RBD operations
if it is only them that cause 'op_rw' counters to be emitted.
Cheers
Mark
[1] Tested using fio and pgbench (database benchmark). I mounted the
volume using the kernel driver (I'll do some more experimentation using
librbd)
Show replies by date
I did say I'd test using librbd - and this changes my observations.
Using fio configured with the rbd driver:
- a random write workload emits about equal 'op_w' and 'op_rw'
initially, then just 'op_w' (until filled in sparse allocation maybe)?
So this certainly does help me understand why I'm seeing a lot of
'op_rw', but any further clarification appreciated!
regards
Mark
On 2/09/20 6:17 pm, Mark Kirkwood wrote:
> Hi,
>
> I'd like to gain a better understanding about what operations emit
> which of these performance counters, in particular when is 'op_rw'
> incremented instead of 'op_r' + 'op_w'?
>
> I've done a little bit of investigation (v12.2.13) , running various
> workoads and operations against an RBD volume (in a cluster with no
> other client activity):
>
> - Most RBD 'operations' (create, rm, features disable/enable, map,
> unmap) emit 'op_rw' and often 'op_w' too
>
> - Program reads and writes against a mounted RBD volume *only* emit
> 'op_r' and 'op_w' (never 'op_rw'), regardless of whether they
are
> 'read + modify' of existing file data (or whether the writes are
> buffered, direct or sync)
>
> Is that correct? Or have I missed a program driven workload that will
> produce 'op_rw'? [1]
>
> In our production clusters I'm seeing similar numbers of 'op_w' and
> 'op_rw' (for a given OSD), which would suggest a lot of RBD operations
> if it is only them that cause 'op_rw' counters to be emitted.
>
> Cheers
>
> Mark
>
> [1] Tested using fio and pgbench (database benchmark). I mounted the
> volume using the kernel driver (I'll do some more experimentation
> using librbd)
>
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
Digging a bit deeper: In my 1st tests in order to mount via kernel I'd
had to disable a number of features on the RBD volume - in particular
'object-map'. So redoing my librbd testing - but disabling various
features immediately after creating the volume I find:
- disabling 'object-map' eliminates all but a handful of 'op_rw'
emitted, so it would appear that maintaining this mapping is what is
causing these [1].
Note I'm not suggesting that 'object-map' is a problem - I'm just
wanting to know what is causing the 'op_rw' counters!
regards
Mark
[1] I had to switch off 'fast-diff' too but this is not relevant to the
'op_r' + 'op_w' vs 'op_rw' discussio.
On 2/09/20 6:49 pm, Mark Kirkwood wrote:
> I did say I'd test using librbd - and this changes my observations.
> Using fio configured with the rbd driver:
>
> - a random write workload emits about equal 'op_w' and 'op_rw'
> initially, then just 'op_w' (until filled in sparse allocation maybe)?
>
> So this certainly does help me understand why I'm seeing a lot of
> 'op_rw', but any further clarification appreciated!
>
> regards
>
> Mark
>
> On 2/09/20 6:17 pm, Mark Kirkwood wrote:
>> Hi,
>>
>> I'd like to gain a better understanding about what operations emit
>> which of these performance counters, in particular when is 'op_rw'
>> incremented instead of 'op_r' + 'op_w'?
>>
>> I've done a little bit of investigation (v12.2.13) , running various
>> workoads and operations against an RBD volume (in a cluster with no
>> other client activity):
>>
>> - Most RBD 'operations' (create, rm, features disable/enable, map,
>> unmap) emit 'op_rw' and often 'op_w' too
>>
>> - Program reads and writes against a mounted RBD volume *only* emit
>> 'op_r' and 'op_w' (never 'op_rw'), regardless of whether
they are
>> 'read + modify' of existing file data (or whether the writes are
>> buffered, direct or sync)
>>
>> Is that correct? Or have I missed a program driven workload that will
>> produce 'op_rw'? [1]
>>
>> In our production clusters I'm seeing similar numbers of 'op_w' and
>> 'op_rw' (for a given OSD), which would suggest a lot of RBD
>> operations if it is only them that cause 'op_rw' counters to be emitted.
>>
>> Cheers
>>
>> Mark
>>
>> [1] Tested using fio and pgbench (database benchmark). I mounted the
>> volume using the kernel driver (I'll do some more experimentation
>> using librbd)
>>
>> _______________________________________________
>> ceph-users mailing list -- ceph-users(a)ceph.io
>> To unsubscribe send an email to ceph-users-leave(a)ceph.io
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io