Hi, So, ran the fio commands, and pasted output (as it's quite a lot) here: https://pastebin.com/KWYEu9uU I hope someone here can draw some conclusions from this output... An observation: We are using proxmox (5.4), and it displays the WEAR level as "N/A", which is unfortunate... :-( Tried upgrading to 6: the same, still no wearout in the GUI. Here is smartctl output:
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.15.18-26-pve] (local build) Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION === Device Model: Micron_5210_MTFDDAK3T8QDE Serial Number: 200426196EEC LU WWN Device Id: 5 00a075 126196eec Firmware Version: D2MU404 User Capacity: 3,840,755,982,336 bytes [3.84 TB] Sector Sizes: 512 bytes logical, 4096 bytes physical Rotation Rate: Solid State Device Form Factor: 2.5 inches Device is: Not in smartctl database [for details use: -P showall] ATA Version is: ACS-3 T13/2161-D revision 5 SATA Version is: SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s) Local Time is: Mon Mar 9 15:26:00 2020 CET SMART support is: Available - device has SMART capability. SMART support is: Enabled
=== START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED
General SMART Values: Offline data collection status: (0x00) Offline data collection activity was never started. Auto Offline Data Collection: Disabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: (13395) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 24) minutes. Conveyance self-test routine recommended polling time: ( 3) minutes. SCT capabilities: (0x0035) SCT Status supported. SCT Feature Control supported. SCT Data Table supported.
SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 100 100 050 Pre-fail Always - 0 5 Reallocated_Sector_Ct 0x0032 100 100 001 Old_age Always - 0 9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 5 12 Power_Cycle_Count 0x0032 100 100 001 Old_age Always - 3 170 Unknown_Attribute 0x0033 100 100 010 Pre-fail Always - 0 171 Unknown_Attribute 0x0032 100 100 000 Old_age Always - 0 172 Unknown_Attribute 0x0032 100 100 001 Old_age Always - 0 173 Unknown_Attribute 0x0032 100 100 000 Old_age Always - 1 174 Unknown_Attribute 0x0032 100 100 000 Old_age Always - 1 183 Runtime_Bad_Block 0x0032 100 100 000 Old_age Always - 0 184 End-to-End_Error 0x0032 100 100 000 Old_age Always - 0 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 188 Command_Timeout 0x0032 100 100 000 Old_age Always - 3 194 Temperature_Celsius 0x0022 071 053 000 Old_age Always - 29 (Min/Max 19/47) 195 Hardware_ECC_Recovered 0x0032 100 100 000 Old_age Always - 0 196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 100 100 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0030 100 100 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x0032 100 100 000 Old_age Always - 0 202 Unknown_SSD_Attribute 0x0030 100 100 001 Old_age Offline - 0 206 Unknown_SSD_Attribute 0x000e 100 100 000 Old_age Always - 0 246 Unknown_Attribute 0x0032 100 100 000 Old_age Always - 1234755968 247 Unknown_Attribute 0x0032 100 100 000 Old_age Always - 40518725 248 Unknown_Attribute 0x0032 100 100 000 Old_age Always - 2271671 180 Unused_Rsvd_Blk_Cnt_Tot 0x0033 001 001 000 Pre-fail Always - 10948 210 Unknown_Attribute 0x0032 100 100 000 Old_age Always - 0
SMART Error Log Version: 1 No Errors Logged
SMART Self-test log structure revision number 1 No self-tests have been logged. [To run self-tests, use: smartctl -t]
SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay.
I have pasted the full "smartctl /dev/sda -P showall" output here: https://pastebin.com/XUxtLA8d The attribute "202 Unknown_SSD_Attribute" is the "Estimated life remaining". Micron's own management tool (msecli, linux version available) works very good. What can be done to make proxmox display the WEAR level? Best, MJ On 06/03/2020 10:53, Marc Roos wrote:
If you are asking, maybe run this?
[global] ioengine=libaio invalidate=1 ramp_time=30 iodepth=1 runtime=180 time_based direct=1 filename=/dev/sdf
[write-4k-seq] stonewall bs=4k rw=write #write_bw_log=sdx-4k-write-seq.results #write_iops_log=sdx-4k-write-seq.results
[randwrite-4k-seq] stonewall bs=4k rw=randwrite #write_bw_log=sdx-4k-randwrite-seq.results #write_iops_log=sdx-4k-randwrite-seq.results
[read-4k-seq] stonewall bs=4k rw=read #write_bw_log=sdx-4k-read-seq.results #write_iops_log=sdx-4k-read-seq.results
[randread-4k-seq] stonewall bs=4k rw=randread #write_bw_log=sdx-4k-randread-seq.results #write_iops_log=sdx-4k-randread-seq.results
[rw-4k-seq] stonewall bs=4k rw=rw #write_bw_log=sdx-4k-rw-seq.results #write_iops_log=sdx-4k-rw-seq.results
[randrw-4k-seq] stonewall bs=4k rw=randrw #write_bw_log=sdx-4k-randrw-seq.results #write_iops_log=sdx-4k-randrw-seq.results
[write-128k-seq] stonewall bs=128k rw=write #write_bw_log=sdx-128k-write-seq.results #write_iops_log=sdx-128k-write-seq.results
[randwrite-128k-seq] stonewall bs=128k rw=randwrite #write_bw_log=sdx-128k-randwrite-seq.results #write_iops_log=sdx-128k-randwrite-seq.results
[read-128k-seq] stonewall bs=128k rw=read #write_bw_log=sdx-128k-read-seq.results #write_iops_log=sdx-128k-read-seq.results
[randread-128k-seq] stonewall bs=128k rw=randread #write_bw_log=sdx-128k-randread-seq.results #write_iops_log=sdx-128k-randread-seq.results
[rw-128k-seq] stonewall bs=128k rw=rw #write_bw_log=sdx-128k-rw-seq.results #write_iops_log=sdx-128k-rw-seq.results
[randrw-128k-seq] stonewall bs=128k rw=randrw #write_bw_log=sdx-128k-randrw-seq.results #write_iops_log=sdx-128k-randrw-seq.results
[write-1024k-seq] stonewall bs=1024k rw=write #write_bw_log=sdx-1024k-write-seq.results #write_iops_log=sdx-1024k-write-seq.results
[randwrite-1024k-seq] stonewall bs=1024k rw=randwrite #write_bw_log=sdx-1024k-randwrite-seq.results #write_iops_log=sdx-1024k-randwrite-seq.results
[read-1024k-seq] stonewall bs=1024k rw=read #write_bw_log=sdx-1024k-read-seq.results #write_iops_log=sdx-1024k-read-seq.results
[randread-1024k-seq] stonewall bs=1024k rw=randread #write_bw_log=sdx-1024k-randread-seq.results #write_iops_log=sdx-1024k-randread-seq.results
[rw-1024k-seq] stonewall bs=1024k rw=rw #write_bw_log=sdx-1024k-rw-seq.results #write_iops_log=sdx-1024k-rw-seq.results
[randrw-1024k-seq] stonewall bs=1024k rw=randrw #write_bw_log=sdx-1024k-randrw-seq.results #write_iops_log=sdx-1024k-randrw-seq.results
-----Original Message----- From: mj [mailto:lists@merit.unu.edu] Sent: 06 March 2020 10:01 To: ceph-users@ceph.io Subject: [ceph-users] Re: Ceph Performance of Micron 5210 SATA?
Last monday I performed a quick test with those two disks already, probably not that relevant, but posting it anyway:
I created a two-disk ceph 'cluster' on just the one local node, and ran the following:
root@ceph:~# rados bench -p scbench 10 write --no-cleanup hints = 1 Maintaining 16 concurrent writes of 4194304 bytes to objects of size 4194304 for up to 10 seconds or 0 objects Object prefix: benchmark_data_ceph_48906 sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 0 0 0 0 0 0 - 0 1 16 107 91 363.961 364 0.0546517 0.155121 2 16 206 190 379.948 396 0.1529 0.159227 3 16 324 308 410.614 472 0.0972163 0.151421 4 16 458 442 441.942 536 0.0484349 0.141799 5 16 590 574 459.141 528 0.0445051 0.136922 6 16 727 711 473.941 548 0.181066 0.134468 7 16 856 840 479.941 516 0.187683 0.133199 8 16 970 954 476.942 456 0.070753 0.132642 9 16 1089 1073 476.831 476 0.193608 0.133754 10 16 1214 1198 479.142 500 0.0999212 0.132529 Total time run: 10.097218 Total writes made: 1215 Write size: 4194304 Object size: 4194304 Bandwidth (MB/sec): 481.321 Stddev Bandwidth: 60.481 Max bandwidth (MB/sec): 548 Min bandwidth (MB/sec): 364 Average IOPS: 120 Stddev IOPS: 15 Max IOPS: 137 Min IOPS: 91 Average Latency(s): 0.132889 Stddev Latency(s): 0.0645579 Max latency(s): 0.336118 Min latency(s): 0.0117049
Do let me know what else you'd want me to do.
MJ _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-leave@ceph.io