Depends on whether it was wired properly. If the switch was placed on the cold wire, the socket will still be hot.

Just wear thick gloves with the pliers.