I have always thought the creepiest locations the show used were the cemeteries and what looked like a cathedral.
Add to that Robert Stack's creep factor and you have some scary stuff!
I thought it was neat how when they filmed at places like Alcatraz, they had it look like guards were still there.