FDCPS Severe Outrages thread

[AMDave]

My pleasure.
No really.
I got 3 primes early on.
None since then, but that's ok because they were good ones
Great to see your post.

[AMDave]

unplanned power outage took the server down for about 5 minutes.
back up and running. tests ok.

[AMDave]

There will be a planned outage for several hours tomorrow while electrical work is carried out.

[AMDave]

Planned outage concluded without incident.
Updates and patches applied before restart.
All good.

[AMDave]

There will be a planned FDCPS outage for further mains electrical work to be carried out.
Planned start is 12 hours from now.
Planned end is 17 hours from now.

[AMDave]

FDCPS planned outage completed on schedule.

[AMDave]

Full DR plan execution and tests for FDCPS completed successfully at 2012-05-20.
Because DR procedures are not 'good' unless you keep verifying that they work.

The FDCPS server PSU fan bearings made some bad noise while at max RPMs during a heat spike about a week ago for a few hours, so it was time to check on a few things.
I made a few updates to the rebuild document during the DR test, but there is one more item outstanding, so I'll commit that back into the project tomorrow.
The project server runs well in a KVM cluster so I'm planning to migrate it to a KVM guest back on the same hardware soon-ish to get all the benefits that brings.
I also verified that the implementation works with Nginx . (very snappy btw)
I'll add that config to the doco for the commit tomorrow.

It went so well I am still looking for the "Gotcha!"