21 Sep
2019
21 Sep
'19
12:22 a.m.
Hi,
I'm testing Ceph with Vmware, using Ceph-iscsi gateway. I reading
documentation* and have doubts some points:
- If I understanded, in general terms, for each VMFS datastore in VMware
will match the an RBD image. (consequently in an RBD image I will possible
have many VMWare disks). Its correct?
- In documentation is this: "gwcli requires a pool with the name rbd, so it
can store metadata like the iSCSI configuration". In part 4 of
"Configuration", have: "Add a RBD image with the name disk_1 in the pool
rbd". In this part, the use of "rbd" pool is a example and I could use any
pool for storage of image, or the pool should be "rbd"?
Resuming: gwcli require "rbd" pool for metadata and I could use any pool
for image, or i will use just "rbd pool" for storage image and metadata?
- How much memory ceph-iscsi use? Which is a good number of RAM?
Regards
Gesiel
* https://docs.ceph.com/docs/master/rbd/iscsi-target-cli/
21 Sep
21 Sep
12:34 a.m.
On Fri, Sep 20, 2019 at 8:55 PM Gesiel Galvão Bernardes
<gesiel.bernardes(a)gmail.com> wrote:
Hi,
I'm testing Ceph with Vmware, using Ceph-iscsi gateway. I reading documentation* and
have doubts some points:
- If I understanded, in general terms, for each VMFS datastore in VMware will match the
an RBD image. (consequently in an RBD image I will possible have many VMWare disks). Its
correct?
yes
- In documentation is this: "gwcli requires a
pool with the name rbd, so it can store metadata like the iSCSI configuration". In
part 4 of "Configuration", have: "Add a RBD image with the name disk_1 in
the pool rbd". In this part, the use of "rbd" pool is a example and I could
use any pool for storage of image, or the pool should be "rbd"?
that's just an example, yes
Resuming: gwcli require "rbd" pool for
metadata and I could use any pool for image, or i will use just "rbd pool" for
storage image and metadata?
you can even store the metadata elsewhere in newer versions, see
options in the config file
- How much memory ceph-iscsi use? Which is a good
number of RAM?
since it can't cache anything: virtually nothing
>
> Regards
> Gesiel
>
> * https://docs.ceph.com/docs/master/rbd/iscsi-target-cli/
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io
12:41 a.m.
Hello Gesiel,
Some iscsi settings are stored in an object, this object is stored in
the rbd pool. Hnece the rbd pool is required.
Your LUN's are mapped to {pool}/{rbdimage}. You should treat these as
you treat pools and rbd images in general.
In smallish deployments I try to keep it simple and make 1 pool for
each deviceclass and make LUN's as big as possible, while still
allowing for setting 1 LUN in maintenance mode in the datastore cluster
within vSphere, in case we need to re-format as part of vmfs upgrade.
Remember to set PSP and recovery timeout properly.
From the LUN level and up, you just treat the storage as any other
iscsi storage connected to vSphere.
the iGW consumes RAM pr. LUN export... I can't remember the default
settings but we are talking about single-digit Gb qith tens of LUN
exported, so it's fairly lightweight.
/Heðin
On frí, 2019-09-20 at 15:52 -0300, Gesiel Galvão Bernardes wrote:
Hi,
I'm testing Ceph with Vmware, using Ceph-iscsi gateway. I reading
documentation* and have doubts some points:
- If I understanded, in general terms, for each VMFS datastore in
VMware will match the an RBD image. (consequently in an RBD image I
will possible have many VMWare disks). Its correct?
- In documentation is this: "gwcli requires a pool with the name rbd,
so it can store metadata like the iSCSI configuration". In part 4 of
"Configuration", have: "Add a RBD image with the name disk_1 in the
pool rbd". In this part, the use of "rbd" pool is a example and I
could use any pool for storage of image, or the pool should be "rbd"?
Resuming: gwcli require "rbd" pool for metadata and I could use any
pool for image, or i will use just "rbd pool" for storage image and
metadata?
- How much memory ceph-iscsi use? Which is a good number of RAM?
Regards
Gesiel
* https://docs.ceph.com/docs/master/rbd/iscsi-target-cli/
_______________________________________________
ceph-users mailing list -- ceph-users(a)ceph.io
To unsubscribe send an email to ceph-users-leave(a)ceph.io
4:43 a.m.
On 09/20/2019 01:52 PM, Gesiel Galvão Bernardes wrote:
Hi,
I'm testing Ceph with Vmware, using Ceph-iscsi gateway. I reading
documentation* and have doubts some points:
- If I understanded, in general terms, for each VMFS datastore in VMware
will match the an RBD image. (consequently in an RBD image I will
possible have many VMWare disks). Its correct?
- In documentation is this: "gwcli requires a pool with the name rbd, so
it can store metadata like the iSCSI configuration". In part 4 of
"Configuration", have: "Add a RBD image with the name disk_1 in the pool
rbd". In this part, the use of "rbd" pool is a example and I could use
any pool for storage of image, or the pool should be "rbd"?
Resuming: gwcli require "rbd" pool for metadata and I could use any pool
for image, or i will use just "rbd pool" for storage image and metadata?
- How much memory ceph-iscsi use? Which is a good number of RAM?
The major memory use is:
1. In RHEL 7.5 kernels and older we allocate max_data_area_mb of kernel
memory per device. The default value for that is 8. You can use gwcli to
configure it. It is allocated when the device is created. In newer
kernels, there is pool of memory and each device can use up to
max_data_area_mb worth of it. The per device default is the same and you
can change it with gwcli. The total pool limit is 2 GB. There is a sysfs
file:
/sys/module/target_core_user/parameters/global_max_data_area_mb
that can be used to change it.
2. Each device uses about 20 MB of memory in userspace. This is not
configurable.
1 Oct
1 Oct
6:02 p.m.
Thank you everyone for the explanations.
Still on this subject: I created an host and attached a disk. For a second
host, for use a shared iscsi storage, I just need add disk to second client?
I tried this:
disk add pool1/vmware_iscsi1
Warning:
'pool1/vmware_iscsi1' mapped to 1 other client(s)
ok
It's a problem, or is correct?
Regards
Gesiel
Em sex, 20 de set de 2019 às 20:13, Mike Christie <mchristi(a)redhat.com>
escreveu:
On 09/20/2019 01:52 PM, Gesiel Galvão Bernardes
wrote:
Hi,
I'm testing Ceph with Vmware, using Ceph-iscsi gateway. I reading
documentation* and have doubts some points:
- If I understanded, in general terms, for each VMFS datastore in VMware
will match the an RBD image. (consequently in an RBD image I will
possible have many VMWare disks). Its correct?
- In documentation is this: "gwcli requires a pool with the name rbd, so
it can store metadata like the iSCSI configuration". In part 4 of
"Configuration", have: "Add a RBD image with the name disk_1 in the pool
rbd". In this part, the use of "rbd" pool is a example and I could use
any pool for storage of image, or the pool should be "rbd"?
Resuming: gwcli require "rbd" pool for metadata and I could use any pool
for image, or i will use just "rbd pool" for storage image and metadata?
- How much memory ceph-iscsi use? Which is a good number of RAM?
The major memory use is:
1. In RHEL 7.5 kernels and older we allocate max_data_area_mb of kernel
memory per device. The default value for that is 8. You can use gwcli to
configure it. It is allocated when the device is created. In newer
kernels, there is pool of memory and each device can use up to
max_data_area_mb worth of it. The per device default is the same and you
can change it with gwcli. The total pool limit is 2 GB. There is a sysfs
file:
/sys/module/target_core_user/parameters/global_max_data_area_mb
that can be used to change it.
2. Each device uses about 20 MB of memory in userspace. This is not
configurable.
9:20 p.m.
On 10/01/2019 07:32 AM, Gesiel Galvão Bernardes wrote:
Thank you everyone for the explanations.
Still on this subject: I created an host and attached a disk. For a
second host, for use a shared iscsi storage, I just need add disk to
second client?
I tried this:
Normally you would do this in a host group.
disk add pool1/vmware_iscsi1
Warning:
'pool1/vmware_iscsi1' mapped to 1 other client(s)
ok
It's a problem, or is correct?
Regards
Gesiel
Em sex, 20 de set de 2019 às 20:13, Mike Christie <mchristi(a)redhat.com
<mailto:mchristi@redhat.com>> escreveu:
On 09/20/2019 01:52 PM, Gesiel Galvão Bernardes wrote:
Hi,
I'm testing Ceph with Vmware, using Ceph-iscsi gateway. I reading
documentation* and have doubts some points:
- If I understanded, in general terms, for each VMFS datastore in
VMware
will match the an RBD image. (consequently in an
RBD image I will
possible have many VMWare disks). Its correct?
- In documentation is this: "gwcli requires a pool with the name
rbd, so
it can store metadata like the iSCSI
configuration". In part 4 of
"Configuration", have: "Add a RBD image with the name disk_1 in
the pool
rbd". In this part, the use of
"rbd" pool is a example and I could use
any pool for storage of image, or the pool should be "rbd"?
Resuming: gwcli require "rbd" pool for metadata and I could use
any
pool
for image, or i will use just "rbd
pool" for storage image and
metadata?
- How much memory ceph-iscsi use? Which is a good number of RAM?
The major memory use is:
1. In RHEL 7.5 kernels and older we allocate max_data_area_mb of kernel
memory per device. The default value for that is 8. You can use gwcli to
configure it. It is allocated when the device is created. In newer
kernels, there is pool of memory and each device can use up to
max_data_area_mb worth of it. The per device default is the same and you
can change it with gwcli. The total pool limit is 2 GB. There is a sysfs
file:
/sys/module/target_core_user/parameters/global_max_data_area_mb
that can be used to change it.
2. Each device uses about 20 MB of memory in userspace. This is not
configurable.
1669
days inactive
1680
days old
5 comments
4 participants
participants (4)
-
Gesiel Galvão Bernardes -
Heðin Ejdesgaard Møller -
Mike Christie -
Paul Emmerich