Albert Ayler was the most primal of the free jazz musicians of the 1960s. He possessed a deep blistering tone—achieved by using the stiffest plastic reeds he could find on his tenor saxophone—and a broad, pathos-filled vibrato that came right out of church music. His trio and quartet records of 1964, like Spiritual Unity and The Hilversum Sessions, show him advancing the improvisational notions of John Coltrane and Ornette Coleman into abstract realms where timbre, not harmony and melody, is the music's backbone. His ecstatic music of 1965 and 1966, like "Spirits Rejoice" and "Truth is Marching In" has been comapred by critics to the sound of a Salvation Army brass band, and involved simple, march-like themes which alternated with wild group improvisations and took jazz back to its pre-Louis Armstrong roots.

Ayler remains something of a cult artist. "Ghosts"—with its bouncy, sing-song melody (rather reminiscent of a nursery rhyme)—is probably his best known tune, and is something of a free jazz standard, having been covered by Lester Bowie, Gary Windo, Eugene Chadbourne, Joe McPhee, John Tchicai and Ken Vandermark, among others.

Saxophonist Mars Williams led a group called Witches and Devils, which was not only named after an Ayler song, but which covered several of his songs.

Peter Brötzmann's Die Like A Dog Quartet is a group loosely dedicated to Ayler. A record called "Little Birds Have Fast Hearts" references Ayler's youthful nickname.

In 2005, guitarist Marc Ribot (who has occasionally performed Ayler's songs for some years) released an album dedicated to the ethic of collective improvisation, entitled Spiritual Unity in honor of Ayler's 1964 album of the same name.

On his 1969 album Folkjokeopus, English guitarist/singer Roy Harper, dedicated the song 'One for All' ("One for Al") to Albert Ayler "who I knew and loved during my time in Copenhagen". Harper considered Ayler to be "one of the leading jazzmen of the age".[3]. Within the Folkejokeopus liner notes Harper states, "In many ways he (Ayler) was the king".

The Albert Ayler Trio's album 'Spiritual Unity' is well known to have been a major influence on Paul McCartney during the recording of The Beatles' celebrated album 'Revolver'.