The Great Southern California Blackout of 2011: 10 Lessons Learned

So here I am at 37,000 feet on my way to DC, and doggone it, no in-flight movies on Southwest. But since I’m sitting in first class, a.k.a., Southwest’s emergency exit aisle, I figured I’d take advantage of all that room and share some of my “Great Southern California Blackout” experiences as the CEO of SDCMS, chair of the San Diego chapter of the Red Cross, and as a private citizen.

1. There was zero warning. None. So, as we used to say in the submarine business, it was a “come as you are war” — you need to be ready all the time. Ditto for earthquakes. Fires and hurricanes give you time to prepare, but not so here.

2. You should do a zero-based assessment of your power systems, as in, what would happen to you if the lights went out and stayed out.

a) What has a backup power supply (sometimes referred to as “uninterruptible power supply” or UPS) connected to it?

b) Does your computer shut down automatically when the UPS says “almost out of battery power”? That option is relatively easy to set up with most state-of-the-art UPSs. And an uncontrolled shutdown is just that, a disaster looking for a place to happen … can you say “blue screen of death”?

c) How will you start the computer back up when the lights go back on? Remotely? In person? Who? How? Etc.

d) What happens when your alarm system runs out of power?

e) Do you test your backup systems? How often? When was the last “end-to-end test” of your “we don’t have any power” plan?

f) Do you have a phone system that does not require 110 volts? FYI, phones were originally designed to be independent of line voltage, and they still work fine with no 110 volt, but you need one of those “oh so yesterday” analog phones. Ditto fax.

g) Where’s your membership data? Could you access it if your server is down? We at SDCMS keep a SECURE (let me emphasize that, SECURE) USB “dongle” disk drive with the entire database of San Diego County physicians, but it’s two-factor encrypted (need to know two passwords and have a gizmo), so its inadvertent loss does not cause your name to be in the papers.

3. What happens to critical systems when the lights go out?

a) Cell phone towers actually worked amazingly well for our 12-hour-plus outage. Probably the batteries would have died had it gone much longer, but I had cellular connectivity the whole time, BUT …

b) How many spare cell phone batteries do you have (I had one, and am buying another two)?

c) Ditto spare batteries for your laptop. I always run around with a spare seven-hour battery.

d) Cell phone texting worked better than cell phone voice.

e) Cars are a wonderful source of power. Do you have a 12 volt DC upconverter to 110 volt AC? I was amazed at the number of people sitting in cars with the engine running going nowhere but charging stuff or (and I wish were kidding about this) listening to their radio, but see 5.a.

f) For those in charge of large systems with external power generators, when was the last time you really “stress tested” your generator? At least one hospital was really embarrassed when their shiny new generator ran for a while then quit. Bad PR.

g) And speaking of generators, consider a handy dandy little generator for “low-power” items — see 3.a and b above and 6 below. Alternatively, you could get one of those hand-crank generators that do the same thing with elbow grease.

4. Do you have a battery-powered radio? So “yesterday” but, guess what, we didn’t, and so we had no clue what was happening. We were able to connect to the Internet via battery-powered systems, and at least one TV station was streaming to the Internet (as opposed to transmitting over the airwaves), and they were a good source of news. After the fact, we figured out that we actually DID have an FM radio — my iPod clone had the capability — we just forgot about it.

5. On a personal note:

a) Gas pumps don’t work without power (duh), so do the thing every good Navy destroyer skipper did: Never ever let gas go below 50 percent (see note 1).

b) ATMs don’t work without power (duh), so how much cash do you have stashed away? How about in the office if you needed to pay for stuff, and credit cards were not accepted?

c) While I don’t usually wear high heels (just don’t look good in ’em), for folks who wear really uncomfortable shoes, you might consider keeping a pair of walking shoes in the trunk of your car. Never know when you might have to walk your way home or off the freeway!

d) Candles really work well (and I wonder what will happen in nine and a half months? ;-), but a copious supply of flashlights and a trunk-load of AA batteries are really nice.

e) Got water? How much? There were parts of San Diego that were under a “boil water” alert.

f) Got food? How much? Didn’t turn out to be a problem because the lights came on in under 24 hours. But what if they hadn’t?

g) And speaking of boiling, that gas-powered barbecue is a surprisingly handy “no power” device. Matches/lighter sticks are mandatory if you want to light a range with no electricity.

6. While there is no sure thing, I was amazed that the Internet providers (DSL and cable) worked IF AND ONLY IF you had 110 volt power that supplied the modem/router. And modems/routers don’t use a lot of power, so I actually purchased a second UPS at home to separately power up my router, modem, phone, and laptop (all low-power users).

7. Satellites don’t care about 110 volt, so we are asking ourselves, what can we get off a satellite? We are considering buying a SAT phone and a satellite-based pager. Pagers … are you kidding? That’s so “yesterday”! Yeah, but two-directional test messaging through satellites worked really well.

8. Do you have a disaster plan for home and work?

9. What’s your plan for manning the office in case of a sustained loss of power? One answer is, “Well no, we’re just going to sit at home.” But for large counties, that may not be an answer. So have you thought about who would man the office, for how long, etc.? And how do you keep from having everyone show up for the first 12 hours, and no one for the next 36?

10. How will you get the word out? What happens if you absolutely cannot? Do you have a plan that allows someone who does have power to get something out?

Please feel free to email any additional observations to Gehring@SDCMS.org. I’ve now participated in three major disasters as CEO — the anthrax scare in 2001, and the San Diego wildfires of 2003 and 2007 — so it’s really not a question of whether, but when!