Gorka was born in Newark, New Jersey in July of 1958. He received his first guitar as a Christmas gift, though Gorka alleges that his older brother stole it from him shortly thereafter. He eventually learned, instead, to play the banjo, and began performing in a folk music group at his church.

Gorka attended Moravian College in Bethlehem, Pennsylvania and joined the Razzy Dazzy Spasm Band which would also include guitarist Richard Shindell. He later began performing solo at the Godfrey Daniels coffee house as the opening act for various musicians that toured there. These included Nanci Griffith, Bill Morrissey, Claudia Schmidt and Jack Hardy.

In 1984, Gorka took first place at the Kerrville Folk Festival. Since then he has toured with artists such as Suzanne Vega, Shawn Colvin, Michael Manring, Christine Lavin, Dave Van Ronk, Cliff Eberhardt, David Massengill, Frank Christian and Lucy Kaplansky.