"Graveyard" is one of Boys for Pele’s ’silly’ little tunes. It’s fairly short and straightforward for a Tori song: she’s going to the graveyard to visit the tomb of a relative. The song is more hopeful than sad: the link between the living and the dead still exists and always will, and the narrator is alive. It’s funny to note "Hungarian Wedding Song," another silly song of the UK Caught A Lite Sneeze maxi single could be seen as "Graveyard"’s counterpart: the protagonist is an engaged woman about to marry her lover, but the last verse seems to reveal she’s pushing up the daisies instead.
This tone is typical of gothic tales and writings of authors such as Edgar Allan Poe, one of Tori’s favorite authors whom she often named among her biggest literary influences. Death and dead people are very present in a lot of his works and Poe also wrote a poem called "Spirits of the Dead", in which a dead person talks to a relative who came to visit his grave and assure him that "The spirits of the dead who stood/In life before thee are again/In death around thee."
The song also more than likely refers to the death of her beloved maternal grandfather of Cherokee descent, Poppa. When he died in January 1973, Tori, nine years old at the time, felt very deeply affected. Tori’s mother confessed to Kalen Rogers in the All These Years biography that she thinks Tori "never got over his death. He was the only person she ever completely respected. She would go to his grave three times a week and sing to him until she was thirteen." Hence the lines "it’s hey to that old man" and "I’m coming to the graveyard to sing you to sleep now". "Frog On My Toe," another Pele B-side, also talks about Poppa and his death.