Why my Linux server ext3 filesystem go read-only?
From my mailbag:
We have 5 Dell server collocated running CentOS 4.x and 5.x server operating system. Sometime my file system (ext3) goes read-only. I’d like to know what could be causing such a problem?
My guess:
a) Hardware problem / hard disk problem, check harddisk for errors.
b) High disk I/O aka busy I/O retry error can mark low level disk call as failed. This will force ext3 to go into read only mode.
c) High disk I/O on SAN
d) SAN is not configured properly for the path failover.
In all sort of problems ext3 goes read-only to protect the filesystem and further damage. If you are using VMWARE, check out official webpage to download SCSI patches or workaround for vmware problems.
So what could be causing the file system on Linux go read-only?
Apart from above generic problem, any other error can trigger filesystem on Linux go read only. I hope our reader / seasoned Linux admin can help to answer this question. Please share the experiences and advice in the comments.
E-mail this to a Friend
Printable Version
Is your site working? Monitor Your Web Site 24/7. Get SMS alerts on server downtime. Free 30-day trial including 20 SMS!
You may also be interested in other helpful articles:
- Shell scripting (BASH) : How to accept password in script
- Setup your own name servers with ISPConfig and GoDaddy domains
- Why doesn’t Linux need defragmenting?
- MySQL Database Runs 60 to 90 Percent Faster on Solaris 10 Than on Red Hat Linux
- History and Culture of Unix Programming - The Art of Unix Programming
Discussion on This Article:
Leave a Reply
We encourage your comments, and suggestions. But please stay on topic, be polite, and avoid spam. Thank you very much for stopping by our site!
Tags: CentOS, check harddisk, disk problem, ext3_filesystem, failover, filesystem, hardware problem, high disk i o, read-only_filesystem, san path failover, vmware
RAID going in degraded mode, the fs may detect that and the server can get in read only.
I have similar issues with RHEL 4ES U3 or U4. Did you get this issue(s) resolved yet? Thanks.
This is a bug in the Linux kernel. It apparently is fixed in kernels as of 2.6.22. Also, I *think* it has been back-ported to some earlier kernels. For example CentOS 5.1 (and RHEL 5.1 for that matter) supposedly have it fixed as well. I was commonly seeing it in CentOS 5 virtual machine running on VMWare. Here is their Knowledge Base article for more info.
Although that article is targeted for virtual machines, apparently the kernel bug affected Linux running on real hardware as well.
OK, so I am having a similar problem. CentOS 4.2. Don’t ask but the following commands were executed:
chown -R 0 etc opt var
chgrp -R 0 etc opt var
chmod -R 0500 opt
chmod -R 0600 etc var
Several things stopped working… after the server was rebooted, I can log in still: root@(none) and I can even see all the etc, opt, and var data. However, when I try:
chmod -R 0755 etc
I get all sorts of changing file blah; Read-only file system. I cannot alter the permissions. Is there a way to fix this?
I ran:
mount -o remount /
That fixed it - no longer in read-only mode. Now I can go correct the damage done.