In an interview with U. Magazine published April 1994, Reznor spoke about the meaning behind the lyrics of this song:
"I was trying to explore some of the paranoia I have as a sexually active person in the age of AIDS," he says of "Heresy." "I guess I feel cheated for not growing up in a more liberated era. At the same time, what gets me mad is the way the right wing has used the 'convenience' of this epidemic in helping to promote their own agenda." 
Best described lyrically as a scathing attack on religion (particularly Western Christianity), "Heresy" contains synthesized bass, simple and powerful drum loops, and distorted guitar riffs backing aggressive vocals and moody atmospherics. The lyrics are ripe with Nietzschean concepts and allude to the infamous "God Is Dead" statement. Like many other songs on the album, the song's bridge contrasts heavily with the verses and choruses by being considerably quieter. This is also the second song on the album to contain "The Downward Spiral Motif" and is played (in power chords at Drop D tuning) by the guitars during the chorus as follows:
Released by Reznor through his remix.nin.com account, this is an unmastered and vocalless version of the album cut.
This remix samples "In Suburbia" and "Television Believers" by Aron Ranen's Religion.
This version splices up the bass track and the drums to create a dancier beat (like the "Slate" remix of "Head Like A Hole"). The choruses are much less layered. On June 22, 2011, Joseph Bishara commented on the creation of the remix on his facebook page: "It was a great time - working at Bryan Adams studio in Vancouver with Dave Ogilvie and Anthony Valcic - programmed around the tracks, and we recorded hours of live at the console mixes, later edited into Heresy and Reptile remixes."
This demo version is found on the 2-disc Deluxe Edition of The Downward Spiral. The vocals in the verses are solely whispered, rather than both whispered and sung in Reznor's high register as on album version. As well, the tempo is slower (105 instead of 115 BPM), and the structure is altered, featuring a differently-arranged introduction.
This song has been played very faithfully to the album version since 1994, though it was considered a setlist rarity until the Wave Goodbye Tour in 2009.
He sewed his eyes shut because he is afraid to see He tries to tell me what I put inside of me He's got the answers to ease my curiosity He dreamed a god up and called it Christianity Your God is dead and no one cares If there is a Hell I will see you there He flexed his muscles to keep his flock of sheep in line He made a virus that would kill off all the swine His perfect kingdom of killing, suffering and pain Demands devotion atrocities done in his name Your God is dead and no one cares Drowning in his own hypocrisy If there is a Hell I will see you there Burning with your god in humility Will you die for this?
The penultimate line of the final chorus as printed in the liner notes is not sung on the final album cut. The last line, rather than being sung at the end of the song, is repeated during the bridge, as evidenced in live performances.