HDDエラー

2010/10/01サーバー構築

11月8日22時35分にHDDでエラーが発生したため一部サービスが停止してしまった。
今回不具合が発生したデバイス(hde2)はデータ領域なのだがここがRead-OnlyになってしまったことでMySQLのデータが書き込めなくなった。
その結果、WebサイトのTOPページ以外は見れなくなっていました。
原因は不明ながらリブートにて復旧しています。

# tail /var/log/kernel.log
Nov  8 22:35:51 bruna kernel: EXT3-fs error (device hde2): ext3_free_blocks: Freeing blocks in system zones - Block = 4194304, count = 1
Nov  8 22:35:51 bruna kernel: Aborting journal on device hde2.
Nov  8 22:35:51 bruna kernel: EXT3-fs error (device hde2) in ext3_free_blocks_sb: Journal has aborted
Nov  8 22:35:51 bruna kernel: EXT3-fs error (device hde2) in ext3_reserve_inode_write: Journal has aborted
Nov  8 22:35:51 bruna kernel: EXT3-fs error (device hde2) in ext3_truncate: Journal has aborted
Nov  8 22:35:51 bruna kernel: EXT3-fs error (device hde2) in ext3_reserve_inode_write: Journal has aborted
Nov  8 22:35:51 bruna kernel: EXT3-fs error (device hde2) in ext3_orphan_del: Journal has aborted
Nov  8 22:35:51 bruna kernel: EXT3-fs error (device hde2) in ext3_reserve_inode_write: Journal has aborted
Nov  8 22:35:51 bruna kernel: EXT3-fs error (device hde2) in ext3_delete_inode: Journal has aborted
Nov  8 22:35:51 bruna kernel: ext3_abort called.
Nov  8 22:35:51 bruna kernel: EXT3-fs error (device hde2): ext3_journal_start_sb: Detected aborted journal
Nov  8 22:35:51 bruna kernel: Remounting filesystem read-only
# tail /var/log/mysql/mysql.err.log
061108 22:35:51  InnoDB: Error: Write to file ./ib_logfile1 failed at offset 0 728064.
061108 22:35:51  InnoDB: Assertion failure in thread 2969328560 in file fil0fil.c line 4023
061108 22:35:52  mysqld restarted
061108 22:35:52 [Warning] Can't create test file /data/mysql/bruna.lower-test
061108 22:35:52 [Warning] Can't create test file /data/mysql/bruna.lower-test
061108 22:35:52  InnoDB: Operating system error number 30 in a file operation.