The first uses of the term were associated with African Americans. Henry notes that "The Mills Brothers learned to harmonize in their father's barber shop in Piqua, Ohio. Several other well-known African American gospel quartets were founded in neighborhood barber shops, among them the New Orleans Humming Four, the Southern Stars and the Golden Gate Jubilee Quartette. Although the Mills Brothers are primarily known as jazz and pop artists and usually performed with instrumental accompaniment, the affinity of their harmonic style with that of the barbershop quartet is clearly in evidence in their music and most notably, perhaps, in their best-known gospel recording, "Jesus Met the Woman at the Well," performed a cappella. Their father founded a barbershop quartet, the Four Kings of Harmony, and the Mills Brothers produced at least three records in which they sang a cappella and performed traditional barbershop material.

Barbershop harmonies remain in evidence in the a cappella music of the black church. The popular, Christian a cappella group Take 6 started in 1980 as The Gentleman's Estate Quartet with the tight, four-part harmony by which barbershop music is known. Early on, the quartet added a fifth harmonic line, but the group's pedigree, like barbershop music, is traceable directly to the black church—and the jazzy renditions of artists like the Mills Brothers, as well.

The revision of a cappella singing was taken up again when a tax lawyer named Owen C. Cash decided that for the art to die out would be a shame. He garnered support from an investment banker called Rupert I. Hall. Both came from Tulsa, Oklahoma. Cash was a true partisan of quartet singing who advertised the fact that he did not want a cappella to fall by the wayside. A meeting was called and at 6.30 pm on Monday April 11, 1938, 26 men gathered on the roof garden of the Tulsa Club in the Alvi Hotel. They eventually burst into four part harmony singing. The police were called and had to ask the participants to "keep it down." The sound of their singing had reached ground level and all traffic stopped to listen, wondering where the harmonic sound was coming from. This incident is spoofed in an episode of the Simpsons.

Cash had struck a chord, albeit unwittingly, and soon, across North America, men responded in their thousands and later in the same year the "Society for the Preservation and Encouragement of Barbershop Quartet Singing in America" was set up, known by the acronym S.P.E.B.S.Q.S.A. at a time when many institutions in the United States were in the habit of using multiple initials to denote their function. More recently, the name was changed to the simpler "Barbershop Harmony Society."