The wives of the super-rich ruled high society with an iron fist after the Civil War. As more and more people became millionaires, though, it was difficult to tell the difference between old money, respectable families, and those who only had vulgar new money. By the 1880s, in order to tell who was acceptable and who wasn’t, the women who were already “in” felt it necessary to create dozens of fashion rules that everyone in the know had to follow. That way, if a woman showed up at the opera in a dress that cost more than most Americans made in a year, but it had the wrong sleeve length, other women would know not to give her the time of day.
Not wearing white outside the summer months was another one of these silly rules. White was for weddings and resort wear, not dinner parties in the fall. Of course it could get extremely hot in September, and wearing white might make the most sense, but if you wanted to be appropriately attired you just did not do it. Labor Day became a federal holiday in 1894, and society eventually adopted it as the natural endpoint for summer fashion.
Not everyone followed this rule. Even some socialites continued to buck the trend, most famously Coco Chanel, who wore white year-round. But even though the rule was originally enforced by only a few hundred women, over the decades it trickled down to everyone else. By the 1950s, women’s magazines made it clear to middle class America: white clothing came out on Memorial Day and went away on Labor Day.
These days the fashion world is much more relaxed about what colors to wear and when, but every year you will still hear people say that white after Labor Day is unacceptable, all thanks to some snobby millionaires over 100 years ago.