How to Fix PostgreSQL Error: Could Not Write Block

The "Could not write block" error occurs when PostgreSQL cannot write a data block to disk due to I/O failures, permissions issues, or storage system problems. This is a critical error indicating potential hardware or filesystem issues.

Impact

This error can cause transaction failures, data corruption, database crashes, and service unavailability. It requires immediate investigation as it often indicates underlying storage problems.

Common Causes

1. Disk full or insufficient space
2. Filesystem errors or corruption
3. Hardware failures (disk, controller, cable)
4. Permission issues on data directory
5. Network storage (NFS, SAN) problems
6. Disk quota exceeded
7. Read-only filesystem
8. I/O timeout or overload

Troubleshooting and Resolution Steps

1. Check disk space and filesystem:

   # Check disk space
   df -h /var/lib/postgresql
   
   # Check filesystem errors
   sudo dmesg | grep -i error
   sudo journalctl -xe | grep -i "I/O error"
   
   # Check filesystem status
   mount | grep postgresql
   

2. Check PostgreSQL logs:

   # View detailed error messages
   sudo tail -100 /var/log/postgresql/postgresql-15-main.log | grep -i "could not write"
   
   # Look for I/O related errors
   sudo grep -i "I/O error\|input/output error\|write failed" /var/log/postgresql/postgresql-15-main.log
   

3. Verify directory permissions:

   # Check PostgreSQL data directory ownership
   ls -ld /var/lib/postgresql/15/main
   # Should be owned by postgres user
   
   # Check permissions
   ls -la /var/lib/postgresql/15/main/
   
   # Fix if needed
   sudo chown -R postgres:postgres /var/lib/postgresql
   sudo chmod 700 /var/lib/postgresql/15/main
   

4. Check for read-only filesystem:

   # Check if filesystem is read-only
   mount | grep /var/lib/postgresql
   
   # If read-only, remount as read-write
   sudo mount -o remount,rw /var/lib/postgresql
   
   # Check why it went read-only (usually hardware issues)
   sudo dmesg | grep -i "read-only"
   

5. Test disk write capability:

   # Test write as postgres user
   sudo -u postgres touch /var/lib/postgresql/15/main/test_write
   sudo -u postgres rm /var/lib/postgresql/15/main/test_write
   
   # Check I/O performance
   sudo -u postgres dd if=/dev/zero of=/var/lib/postgresql/testfile bs=1M count=100
   sudo rm /var/lib/postgresql/testfile
   

6. Check for disk errors:

   # Check disk health (SMART)
   sudo smartctl -a /dev/sda
   
   # Check for bad blocks
   sudo badblocks -v /dev/sda
   
   # System logs for disk errors
   sudo dmesg | grep -i "ata\|sda\|scsi"
   

7. Check disk quotas:

   # Check if quotas are enabled
   quota -v
   
   # Check postgres user quota
   sudo quota -u postgres
   
   # Increase quota if needed
   sudo setquota -u postgres 0 0 0 0 /var/lib/postgresql
   

8. For network storage (NFS):

   # Check NFS mount status
   mount | grep nfs
   df -h | grep nfs
   
   # Test NFS connectivity
   showmount -e nfs-server-ip
   
   # Remount if needed
   sudo umount /var/lib/postgresql
   sudo mount -t nfs nfs-server:/export /var/lib/postgresql
   
   # Check NFS mount options in /etc/fstab
   # Recommended: hard,intr,rsize=32768,wsize=32768
   

9. Recover from corruption:

   # Stop PostgreSQL
   sudo systemctl stop postgresql
   
   # Run filesystem check (if possible)
   # Requires unmounting - may need single-user mode
   sudo fsck -f /dev/sda1
   
   # Check PostgreSQL data integrity
   sudo -u postgres /usr/lib/postgresql/15/bin/pg_resetwal --dry-run /var/lib/postgresql/15/main
   
   # If necessary (LAST RESORT - can cause data loss)
   # sudo -u postgres /usr/lib/postgresql/15/bin/pg_resetwal -f /var/lib/postgresql/15/main
   
   # Start PostgreSQL
   sudo systemctl start postgresql
   
   # Run integrity checks
   sudo -u postgres psql -c "SELECT * FROM pg_database;" postgres
   

10. Monitor I/O performance:

    # Monitor disk I/O
    iostat -x 5
    
    # Monitor specific process I/O
    sudo iotop -o
    
    # Check PostgreSQL I/O statistics
    sudo -u postgres psql << EOF
    SELECT * FROM pg_stat_bgwriter;
    SELECT * FROM pg_stat_database;
    EOF
    

Additional Information

• Always investigate underlying cause - often hardware related
• Regular backups are critical for recovery
• Monitor disk health proactively (SMART)
• Use reliable storage systems for production
• Consider RAID for redundancy
• Test disaster recovery procedures
• Keep system logs for troubleshooting
• Network storage requires stable, low-latency connections

Frequently Asked Questions

Q: Can I safely ignore this error if it's transient?
A: No, investigate immediately. Even transient write errors indicate potential hardware or storage issues that can lead to data loss.

Q: Will this error cause data corruption?
A: Possibly. PostgreSQL uses WAL to prevent corruption, but repeated write failures can lead to inconsistencies. Verify data integrity after resolving.

Q: How do I check if my disk is failing?
A: Use smartctl -a /dev/sda to check SMART status. Look for reallocated sectors, pending sectors, or uncorrectable errors.

Q: Can network issues cause this error?
A: Yes, if PostgreSQL data is on network storage (NFS, iSCSI). Network interruptions can appear as write failures.

Q: Should I continue running if I see this error occasionally?
A: No, resolve the underlying issue immediately. Continuing operation risks data corruption or loss.

Q: How do I recover if PostgreSQL won't start after this error?
A: Check logs, verify filesystem integrity, ensure proper permissions, and potentially restore from backup if corruption occurred.

Q: Can insufficient memory cause this error?
A: Indirectly - memory pressure can lead to excessive I/O and timeouts, but the error itself indicates actual write failure.

Q: Is RAID sufficient protection against this error?
A: RAID provides redundancy but doesn't prevent all write errors. Still need monitoring, backups, and proper maintenance.