Salsa Again the doc shows and we have used layering only as a feature for iSCSI Further down it gives you specific settings for the luns/images In our case we let vmware/veeam snapshot and make copies of our VMs There is a new Beta of SES that bypasses the iscsi gateways for Windows servers We have not had a chance to look at it yet, but it looks very interesting Thanks Joe

...

...

Salsa <salsa@protonmail.com> 9/3/2020 9:03 AM >>> Joe,

sorry, I should have been clearer. The incompatible rbd features are exclusive-lock, journaling, object-map and such. The info comes from here: https://documentation.suse.com/ses/6/html/ses-all/ceph-rbd.html -- Salsa Sent with ProtonMail Secure Email. ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Thursday, September 3, 2020 12:58 PM, Joe Comeau <Joe.Comeau@hli.ubc.ca> wrote:

Here is a link for iSCSI/RBD implementation guide from SUSE for this year for vmware (Hyper-v should be similar)

https://www.suse.com/media/guide/suse-enterprise-storage-implementation-guid...

We've been running rbd/iscsi for 4 years

Thanks Joe

...

...

...

Salsa salsa@protonmail.com 9/2/2020 3:08 PM >>>

I just came across a Suse documentation stating that RBD features are

not iSCSI compatible. Since I had 2 cases of image corruption in this scenario in 10 days I'm wondering if my setup is to blame.

So question is if it is possible to provide disks to a Windows Server

2019 via iSCSI while using rbd-mirror to backup data to a second cluster? I created all images with all features enabled. Is that compatible?

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Salsa

ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io

ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io

_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io

I did the same, 1 or 2 years ago, creating a replicated_ruleset_hdd and replicated_ruleset_ssd. Eventhough I did not have any ssd's on any of the nodes at that time, adding this hdd type criteria made pg's migrate. I thought it was strange that this happens on a hdd only cluster, so I mentioned it here. I am not sure however if this is still an issue, but better take this into account. -----Original Message----- To: ceph-users@ceph.io Subject: [ceph-users] New pool with SSD OSDs Hello! We have a Ceph cluster with 30 HDD 4 TB in 6 hosts, only for RBD. Now, we're receiving other 6 servers with 6 SSD 2 TB each and we want to create a separate pool for RBD on SSD, and let unused and backup volumes stays in HDD. I have some questions: As I am only using "replicated_rule". ¿If I add an SSD OSD to the cluster, Ceph starts to migrate PGs to it? If so, to prevent this, first I have to create rule like     # ceph osd crush rule create-replicated pool-hdd default host hdd and then     #ceph osd pool set rbd crush_rule pool-hdd ? Or, if Ceph does not mix automatically hdd and ssd, I create the SSD OSD and then     # ceph osd crush rule create-replicated pool-ssd default host ssd     # ceph osd pool create pool-ssd 256 256 ssdpool ? And then migrate images from one to another pool as needed. Any thoughts are wellcome! Thanks in advanced for your time. Javier.-

Exactly! I created a replicated-hdd rule and set it to an existing small pool without any changes on OSDs (all HDD) and PGs starts migration... It seems like new rules forces migrations... El 14/9/20 a las 11:09, André Gemünd escribió:

Same happened to us two weeks ago using nautilus, although we added the rules and storage classes.

----- Am 14. Sep 2020 um 16:02 schrieb Marc Roos M.Roos@f1-outsourcing.eu:

...

I did the same, 1 or 2 years ago, creating a replicated_ruleset_hdd and replicated_ruleset_ssd. Eventhough I did not have any ssd's on any of the nodes at that time, adding this hdd type criteria made pg's migrate. I thought it was strange that this happens on a hdd only cluster, so I mentioned it here. I am not sure however if this is still an issue, but better take this into account.

-----Original Message----- To: ceph-users@ceph.io Subject: [ceph-users] New pool with SSD OSDs

Hello!

We have a Ceph cluster with 30 HDD 4 TB in 6 hosts, only for RBD.

Now, we're receiving other 6 servers with 6 SSD 2 TB each and we want to create a separate pool for RBD on SSD, and let unused and backup volumes stays in HDD.

I have some questions:

As I am only using "replicated_rule". ¿If I add an SSD OSD to the cluster, Ceph starts to migrate PGs to it?

If so, to prevent this, first I have to create rule like

    # ceph osd crush rule create-replicated pool-hdd default host hdd

and then

    #ceph osd pool set rbd crush_rule pool-hdd

?

Or, if Ceph does not mix automatically hdd and ssd, I create the SSD OSD and then

    # ceph osd crush rule create-replicated pool-ssd default host ssd

    # ceph osd pool create pool-ssd 256 256 ssdpool

?

And then migrate images from one to another pool as needed.

Any thoughts are wellcome!

Thanks in advanced for your time.

Javier.-

_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io

Oh, yes, I think this would have helped indeed! Thanks for pointing it out. Greetings André ----- Am 14. Sep 2020 um 16:48 schrieb Stefan Kooman stefan@bit.nl:

On 2020-09-14 16:09, André Gemünd wrote:

...

Same happened to us two weeks ago using nautilus, although we added the rules and storage classes.

I think this post [1] from Wido den Hollander might be useful information. That way you can avoid data movement if data is already on hdd.

Gr. Stefan

[1]: https://blog.widodh.nl/2019/02/comparing-two-ceph-crush-maps/

-- Dipl.-Inf. André Gemünd, Leiter IT / Head of IT Fraunhofer-Institute for Algorithms and Scientific Computing andre.gemuend@scai.fraunhofer.de Tel: +49 2241 14-2193 /C=DE/O=Fraunhofer/OU=SCAI/OU=People/CN=Andre Gemuend

On 2020-09-14 17:51, Tecnologia Charne.Net wrote:

Thanks Stefan! Compiling crush map by hand on production cluster makes me sweat.... but we like to take risks, don't we?

I the crushtool says it's OK, I guess it's OK ;-). But yeah, that's the most powerful operation one can perform on a cluster I guess ... you don't want to mess that up. Gr. Stefan