Way way back!

So we had some old backups from when I moved the site into hostmonster…..The steps to recovery went something like this:

1. Resurrect old file server that had stopped working 7 months ago that you had been putting off fixing
2. Repair broken linux md software RAID array
3. Update server to modern linux patches
4. Write backup scripts for hostmonster to download daily to repaired server
5. Bother wife to go through photos from 2003-2008 while 4000 miles away to make room for your new daily backups (Bad idea)
6. Find out you had plenty of unallocated space to backup the data all along
7. Discover your mysql database backup is from a version of wordpress not available since 2007
8. Hit head against computer for several hours trying to make the database format work anyway
9. Discover there are old version of wordpress still available that you could install and then recovery your database into for possible additional export methods.
10. Guess and check which wordpress database schema version the mysql backup might have come from
11. Guess wrong
12. Guess again
13. Guess wrong
14. Guess again
15. Guess right!
16. Export posts and comments from old wordpress blog
17. Reinstall current wordpress version since you screwed up your install trying hammer your square database backups into the newer round database hole.
18. Import old content from your export file that you created
19. Tweak blog settings (again)
20. Delete old wordpress version
21. Delete old databases
21. Post about your joy!

A Crummy Day

So on Thursday morning I woke up like most mornings and on my way out the door, checked my email for a message from my wife on oatmail.org.  The server was replying with a message indicating “invalid username or password.”  This was funny.  I don’t have any password expiration policies set on oatmail accounts.

All day on Thursday I was up in Portland at a vendor event, and I wasn’t able to look into the errors at all. When I got home to my house in Eugene I discovered that all of our data had been deleted from my hosting providers’ storage.   This included the last 6 years of mail (for some users it was more like 9 years of email).  It also included all of our files and this blog that has chronicled our endeavors together since marriage.

When I discovered that all of our account data had suddenly gone missing, I contacted technical support immediately.  They claimed that the data had been deleted by me.  I’m the only person with direct login to the account and it turns out I use a unique password for the main account that nobody else knows or is used elsewhere on the web.  I also hadn’t logged into the account in over 5 weeks so this was a bit of a quandry for me.

I pushed the hosting company for logs or other data that could backup their accusation and they were unable to produce any information at all showing that anyone had accessed the account.  Despite this, they insisted that they did not delete our account data.  Because of their lack of explanation, I can only assume that our account was compromised and some disgruntled hacker decided to delete the last 9 years of our lives.  The hosting provider refuses to investigate the ticket as a security incident.

Adding to the insult, they had no backups of our account.  The data is gone.  I should have assumed that they wouldn’t take backups.  This is my bad, and a valuable less that I should have already learned.

I have re-created the blog, I’ll see if there is some magical way to restore the content from internet archives at a later date.  I’m sure Rachel will post something soon on the website, she is traveling in Japan at the moment.

In any case, sorry for those of you who couldn’t get in touch with us over the weekend, our email should be working again.  I have also changed some configuration options to remove access for “service accounts” that hostmonster creates by default, I have purchased SSL certificates for our email websites, and I am implementing online nightly backups of the website.  All of these things I should have done before.  I can’t help suspect though, that this was related to a mess-up by the hosting provider rather than an actual security compromise of our account.  Either way, I’ll never know.