One reported it comes on if you short the base and the center contact. Another liked the fact that it does not come on if the power fails but it was already switched off.

So, best guess: when the power goes out, there is no actual break in the circuit*. There is a closed loop to the nearest transformer. The tip and ring of the bulb can still "see" each other. If that condition holds, but no line voltage is detected => bulb comes on. Flip the switch => tip and ring lose contact => bulb stays off.

* Well, rarely. Unless it comes with a side order of fried squirrel because someone didn't bury the line between the pole and the house properly...