System Status

Incident history

• Disk usage on shared cluster 3: 61%

  Right now · We have noticed. We are deciding what to do about it.
  Investigating
• Nightly backup job has not completed successfully in 6 days

  Ongoing since Tuesday · It keeps retrying automatically. We keep letting it.
  Ongoing
• One VPS host running at 68°C

  Since this morning · We've pointed a fan at the hard drives. Someone will look at the thermals properly, eventually.
  Monitoring
• Unable to send email to recipients more than 500 miles away

  Reported yesterday · Currently failing at 412 miles. We are aware this sounds impossible. We are looking into it anyway.
  Investigating
• SSL certificate expired for 6 hours

  Last month · Noticed after the third forwarded "your connection is not private" email. Renewed shortly after.
  Resolved
• Ran fsck on a volume that was making concerning noises

  2 months ago · It came back clean, which surprised everyone involved.
  Resolved
• Coffee machine unresponsive

  This morning · Unplugged it, plugged it back in. Unrelated to hosting, but it worked.
  Resolved
• Someone asked about our uptime percentage

  Last week · Changed the subject.
  Resolved
• A server made a noise

  Last month · It stopped making the noise on its own. We consider this a win.
  Resolved
• Brief period of everything working perfectly

  3 months ago · Concerning, but it passed.
  Resolved

A few of the above are marked as ongoing. We are aware of them, which we consider a meaningful first step.

* Yes, this really happened to someone else, once. The original story.