死にかけの2TB HDDを使っていたらついに死亡した
9 min read
こんにちは、無能です。
こないだ3TBのHDDが死んでから復旧したのですが、今度は2TBのHDDが死にました。
そのメモです。
ちなみに壊れてしょげているのかといえばちょっと悲しいですけどそもそもそんな大したデータを入れている訳じゃなく、あくまでたくさんあるPCのなかでのファイル共有だとかをしていたりで元のファイルはまた別のところにあります。なのでそんなしょげて無いってことです。
悲しいもんは悲しいですが、元々不良セクタありで500円くらいで買ったものである意味壊すために買ったみたいなものです。
いざ壊れてみないとどうやれば良いかわからないのでいつだって自分が実験台です。
ちなみに、すでに明らかにおかしい異音がするのでもはやデータ取ってこれたら奇跡ですね。
前回は無事に成功して復旧し再度自宅鯖でしごき上げています。
よく言う、ドMはドSの素質を持つ、そんなところです。
スーパーブロックの確認
$ sudo mkfs.ext4 -n /dev/sdb
mke2fs 1.47.1 (20-May-2024)
Creating filesystem with 122094657 4k blocks and 30531584 inodes
Filesystem UUID: 81d0a0ba-490a-4ea0-bb08-ee84f122b21e
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000
ファイルシステムが保有するメタデータ的なもののバックアップが確認出来ます。
スーパーブロックの復元
もし復元したい場合 :
sudo e2fsck -b 71663616 /dev/sdb1
-bの引数にSuperblock backups stored on blocksの値を入れます。
smartctl
ディスクのステータス確認
sudo smartctl -a /dev/sdb
実行結果
$ sudo smartctl -a /dev/sdb
smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.11.1-artix1-1] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Family: Seagate Barracuda Green (AF)
Device Model: ST2000DL003-9VT166
Serial Number: 5YD3ZLT5
LU WWN Device Id: 5 000c50 038d5b835
Firmware Version: CC32
User Capacity: 2,000,398,934,016 bytes [2.00 TB]
Sector Size: 512 bytes logical/physical
Rotation Rate: 5900 rpm
Device is: In smartctl database 7.3/5528
ATA Version is: ATA8-ACS T13/1699-D revision 4
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 1.5 Gb/s)
Local Time is: Thu Oct 17 00:58:11 2024 JST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART Status not supported: Incomplete response, ATA output registers missing
SMART overall-health self-assessment test result: PASSED
Warning: This result is based on an Attribute check.
See vendor-specific Attribute list for marginal Attributes.
General SMART Values:
Offline data collection status: (0x82) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 612) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 1) minutes.
Extended self-test routine
recommended polling time: ( 334) minutes.
Conveyance self-test routine
recommended polling time: ( 2) minutes.
SCT capabilities: (0x30b7) SCT Status supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 098 079 006 Pre-fail Always - 50824899
3 Spin_Up_Time 0x0003 096 092 000 Pre-fail Always - 0
4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 554
5 Reallocated_Sector_Ct 0x0033 047 047 036 Pre-fail Always - 35272
7 Seek_Error_Rate 0x000f 086 060 030 Pre-fail Always - 434289978
9 Power_On_Hours 0x0032 088 088 000 Old_age Always - 11048
10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0
12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 554
183 Runtime_Bad_Block 0x0032 100 100 000 Old_age Always - 0
184 End-to-End_Error 0x0032 098 098 099 Old_age Always FAILING_NOW 2
187 Reported_Uncorrect 0x0032 001 001 000 Old_age Always - 9017
188 Command_Timeout 0x0032 099 098 000 Old_age Always - 47245361169
189 High_Fly_Writes 0x003a 001 001 000 Old_age Always - 342
190 Airflow_Temperature_Cel 0x0022 061 055 045 Old_age Always - 39 (Min/Max 39/39)
191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0
192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 511
193 Load_Cycle_Count 0x0032 100 100 000 Old_age Always - 576
194 Temperature_Celsius 0x0022 039 045 000 Old_age Always - 39 (0 22 0 0 0)
195 Hardware_ECC_Recovered 0x001a 036 005 000 Old_age Always - 50824899
197 Current_Pending_Sector 0x0012 001 001 000 Old_age Always - 13032
198 Offline_Uncorrectable 0x0010 001 001 000 Old_age Offline - 13032
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0
240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 11007 (95 12 0)
241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 4111343507
242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 2489096598
SMART Error Log Version: 1
ATA Error Count: 4459 (device log contains only the most recent five errors)
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.
Error 4459 occurred at disk power-on lifetime: 11048 hours (460 days + 8 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
25 d5 b8 ff ff ff ef 00 00:01:07.241 READ DMA EXT
25 d5 b8 ff ff ff ef 00 00:00:57.948 READ DMA EXT
25 d5 98 ff ff ff ef 00 00:00:57.934 READ DMA EXT
25 d5 a0 ff ff ff ef 00 00:00:57.919 READ DMA EXT
25 d5 a0 ff ff ff ef 00 00:00:57.914 READ DMA EXT
Error 4458 occurred at disk power-on lifetime: 11048 hours (460 days + 8 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
25 d5 b8 ff ff ff ef 00 00:00:57.948 READ DMA EXT
25 d5 98 ff ff ff ef 00 00:00:57.934 READ DMA EXT
25 d5 a0 ff ff ff ef 00 00:00:57.919 READ DMA EXT
25 d5 a0 ff ff ff ef 00 00:00:57.914 READ DMA EXT
25 d5 a0 ff ff ff ef 00 00:00:57.899 READ DMA EXT
Error 4457 occurred at disk power-on lifetime: 11048 hours (460 days + 8 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
25 d5 c0 ff ff ff ef 00 00:00:30.313 READ DMA EXT
25 d5 08 ff ff ff ef 00 00:00:30.312 READ DMA EXT
25 d5 50 ff ff ff ef 00 00:00:30.296 READ DMA EXT
25 d5 58 ff ff ff ef 00 00:00:30.294 READ DMA EXT
25 d5 58 ff ff ff ef 00 00:00:30.292 READ DMA EXT
Error 4456 occurred at disk power-on lifetime: 11048 hours (460 days + 8 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
25 d5 f0 ff ff ff ef 00 00:00:25.557 READ DMA EXT
25 d5 f0 ff ff ff ef 00 00:00:25.551 READ DMA EXT
25 d5 10 ff ff ff ef 00 00:00:25.551 READ DMA EXT
25 d5 20 ff ff ff ef 00 00:00:25.549 READ DMA EXT
25 d5 38 ff ff ff ef 00 00:00:25.548 READ DMA EXT
Error 4455 occurred at disk power-on lifetime: 11048 hours (460 days + 8 hours)
When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455
Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
25 d5 08 ff ff ff ef 00 00:00:56.960 READ DMA EXT
25 d5 10 ff ff ff ef 00 00:00:56.958 READ DMA EXT
25 d5 f0 ff ff ff ef 00 00:00:56.953 READ DMA EXT
25 d5 08 ff ff ff ef 00 00:00:56.947 READ DMA EXT
25 d5 10 ff ff ff ef 00 00:00:56.945 READ DMA EXT
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Short offline Completed: read failure 70% 11048 436275968
# 2 Short offline Completed: read failure 70% 11048 436209928
# 3 Short offline Interrupted (host reset) 00% 11048 -
# 4 Short offline Completed: read failure 70% 11048 436209928
# 5 Short offline Completed: read failure 90% 11047 436209928
# 6 Short offline Completed: read failure 90% 11047 436209928
# 7 Short offline Completed: read failure 90% 11047 436209928
# 8 Short offline Interrupted (host reset) 00% 11047 -
# 9 Short offline Completed: read failure 90% 11047 436209928
#10 Short offline Interrupted (host reset) 00% 11046 -
#11 Short offline Completed: read failure 90% 11046 1438650248
#12 Short offline Interrupted (host reset) 00% 11045 -
#13 Short offline Completed: read failure 10% 10835 266787488
#14 Short offline Completed: read failure 10% 10667 266787488
#15 Short offline Completed: read failure 60% 10499 266787488
#16 Short offline Completed: read failure 10% 10331 266787488
#17 Short offline Completed: read failure 10% 10163 266787488
#18 Short offline Completed: read failure 10% 9995 266765001
#19 Short offline Completed: read failure 10% 9982 266765001
#20 Short offline Aborted by host 10% 9981 -
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
The above only provides legacy SMART information - try 'smartctl -x' for more
簡易テスト
実際には最初に-aで確認したあとにテストしてもう一度実行が正しい確認方法かと思います。
sudo smartctl -t short /dev/sdb1
longを指定すると60分くらい掛かるらしい。
badblocksコマンドすら効かないときに不良セクタをfsckに渡したい
不良ブロックを検出するだけなら便利なbadblocksコマンドが存在する。
あくまでこれはあるかどうかを見るだけなので修復ではなく、fsckコマンドのオプションでbadblocksも実行して修復するオプションも存在する。
結局sata接続なら良いのかもしれないが、USB-HDDのreadすら出来ない不良ブロックがあった場合は自動でこんな感じに接続がリセットされて続行できない。
[ 2606.101954] sd 6:0:0:0: [sdb] tag#0 CDB: Read(10) 28 00 1a 00 09 18 00 00 e8 00
[ 2606.101960] blk_print_req_error: 9 callbacks suppressed
[ 2606.101965] I/O error, dev sdb, sector 436209944 op 0x0:(READ) flags 0x80700 phys_seg 3 prio class 0
[ 2636.680192] usb 3-2: reset high-speed USB device number 25 using xhci_hcd
[ 2640.153840] usb 3-2: USB disconnect, device number 25
自分のようなディスク自体の損傷が激しすぎる場合、SATA接続でPC起動させようとするものならBIOSがディスクを見つけれないのかBIOS起動前にスタックする。
これはSSDでも同じ現象が起きている。
ファイル読み込みから不良セクタ情報を渡す
結局このようにカーネルから出ている情報を元にテキストファイルにread line出来るような形式で記述していく。
dmesg
[ 3140.292122] sd 6:0:0:0: [sdb] tag#0 CDB: Read(10) 28 00 1a 00 09 48 00 00 b8 00
[ 3140.292130] I/O error, dev sdb, sector 436209992 op 0x0:(READ) flags 0x80700 phys_seg 2 prio class 0
badblockの情報として渡したいファイル
$ cat badblock
436209984
436275968
436209992
そしてfsckでこれを渡す
sudo e2fsck -f -y -l ./badblock /dev/sdb1
なんか気楽なノリで記述しているが相当危険であることは間違いないのでやらないほうがいい。
おわりに
復旧するコマンドとして、ddrescueもあるが多分Sata接続が出来て完全にHDDの物理的破損でなければデータ救出は出来ると思う。
自分みたいな明らかにおかしいディスクはごく稀なのであまり参考にはならないはず。実験としてディスクで遊びたい人向け。