He spent his early years in the Brooklyn neighborhood of East New York, where he has been rapping at block parties since the early days of his youth. He first showcased his hardcore Brooklyn style to audiences on "I'm the Man," a track from Gang Starr's 1992 album Daily Operation. The following year he released his first single, "Come Clean", which was produced by DJ Premier and became an instant underground hit. His first album, The Sun Rises in the East, released in 1994, and produced entirely by DJ Premier, is considered a classic, and was one of the most acclaimed Hip Hop albums of its time.

As an interesting side note, the album cover shows the World Trade Center on fire only one year after the 1993 bombing of the North Tower. The album featured Jeru's signature conscious lyrics, however he was criticized by The Fugees for his explicit lyrics particularly, in the song "Da Bichez". Fugees member Pras lightly mentioned Jeru on the track "Zealots", from the group's landmark 1996 album The Score, with the line "No matter who you damage, you're still a false prophet", referencing Jeru's single "You Can't Stop the Prophet". Jeru lightly responded in the intro to the track Me or The Papes.

Jeru followed up in 1996 with his sophomore album Wrath of the Math again produced solely by DJ Premier. The album was also widely acclaimed, though not on the same level as his debut. Like on his first album, Jeru was accusatory of commercial hip hop artists and record labels (Death Row, Bad Boy), who he criticized on the concept track "One Day". After the release of Wrath of the Math, Jeru had a falling out with DJ Premier and Guru, and was missing from the scene until 1999, when he released his third album, Heroz4hire, released together with Mizmarvel. Heroz was his first album without production from DJ Premier and also his first under his then newly created KnowSavage Records. It featured the single "99.9 Pa Cent", which was a verbal attack on his former affiliates Gang Starr. Heroz4hire was less critically acclaimed, due mostly to Premier's absence. His latest album, Divine Design, released in 2003, was the first album under his new record label, Ashenafi Records. The album received little attention and mixed reviews. A new Jeru album, tentatively titled Still Rising, has been announced, currently with no release date.