Analysis: Is it Presidents' Day, President's Day or Presidents Day?

Some American federal holidays are fairly simple and celebrated by everyone, such as Independence Day and Thanksgiving. Others are controversial, like Columbus Day. And then there is the case of the third Monday of February.