It seems all the big vendors feel 2x is safe with NVMe
but
I get the feeling this community feels otherwise
Definitely!
As someone who works for a big vendor (and I have since I worked at
Fusion-IO way back in the old days), IMO the correct way to phrase
this would probably be that "someone in technical marketing at the big
vendors" was convinced that 2x was safe enough to put in a white paper
or sales document. They (we, I guess, since I'm one of these types of
people) are focused on performance and cost numbers and as much as I
hate to admit it, it can get in the way of long-term reliability
settings sometimes.
This doesn't mean that they are "wrong" -- these documents are
primarily meant to show the capabilities of their hardware, with a
bill of materials containing their part numbers. It is expected that
end users will adjust a few things when it comes to a production
environment.
The idea that NVMe is safer than spinning rust drives is not
necessarily true -- and it's beside the point. You are just as likely
to run into a weird situation where an OSD or pg acts up or disappears
for non-hardware reasons.
Unless you can live with "nine fives" instead of "five nines" (say, a
caching type of application where you can re-generate the data), use a
size of at least 3 -- and if you can't afford this much storage then
look at erasure coding schemes.
All of this is IMO of course,
Mark
On Thu, Feb 4, 2021 at 4:38 AM Adam Boyhan <adamb(a)medent.com> wrote:
>
> I know there is already a few threads about 2x replication but I wanted to start one
dedicated to discussion on NVMe. There are some older threads, but nothing recent that
addresses how the vendors are now pushing the idea of 2x.
>
> We are in the process of considering Ceph to replace our Nimble setup. We will have
two completely separate clusters at two different sites that we are using rbd-mirror
snapshot replication. The plan would be to run 2x replication on each cluster. 3x is still
an option, but for obvious reasons 2x is enticing.
>
> Both clusters will be spot on to the super micro example in the white paper below.
>
> It seems all the big vendors feel 2x is safe with NVMe but I get the feeling this
community feels otherwise. Trying to wrap my head around were the disconnect is between
the big players and the community. I could be missing something, but even our Supermicro
contact that we worked the config out with was in agreement with 2x on NVMe.
>
> Appreciate the input!
>
> [
https://www.supermicro.com/white_paper/white_paper_Ceph-Ultra.pdf |
https://www.supermicro.com/white_paper/white_paper_Ceph-Ultra.pdf ]
>
> [
https://www.redhat.com/cms/managed-files/st-micron-ceph-performance-referen…
]
> [
https://www.redhat.com/cms/managed-files/st-micron-ceph-performance-referen…
|
https://www.redhat.com/cms/managed-files/st-micron-ceph-performance-referen…
]
>
> [
https://www.samsung.com/semiconductor/global.semi/file/resource/2020/05/red…
|
https://www.samsung.com/semiconductor/global.semi/file/resource/2020/05/red…
]
> _______________________________________________
> ceph-users mailing list -- ceph-users(a)ceph.io
> To unsubscribe send an email to ceph-users-leave(a)ceph.io