The story opens with John O’Donnel and his friend John Conrad being awakened at night by frantic pounding on the door. Job Kiles, a miserly and formerly composed neighbor, bursts in screaming that he has seen his twin brother Jonas—who has been dead for a week—leering in at his window. Job is convinced Jonas has returned from Hell to drag him down.

Conrad, ever the rationalist, dismisses this as madness, but agrees to accompany Job and O’Donnel to Jonas’s tomb in the Dagoth Hills to reassure him. As they traverse the windswept hills, Job raves about Jonas's long travels in the Orient and accuses him of being a vampire who dabbled in dark magic. He reveals Jonas chose to be buried in an ancient tomb called Pirate Hill, built centuries ago by a buccaneer ancestor, Captain Jacob Kiles.

They reach the tomb. The door has not been opened, and there are no signs of recent disturbance. Still, Job insists on going in alone, armed with a flashlight and a stake. Moments later, they hear a blood-curdling scream. Job bursts out of the tomb, collapses, and dies in convulsions, his face contorted in horror. There is no wound, no physical cause of death—only the terror of something he saw.

Conrad and O’Donnel inspect the tomb. Jonas’s coffin is empty. But then they discover a secret pivoting stone in the rear wall. Behind it lies a tunnel carved into the rock. Believing Jonas to have faked his death using Eastern cataleptic techniques, Conrad speculates Jonas planned to murder his brother and assume his life and identity. They enter the tunnel and find a subterranean network of corridors and chambers.

Deeper inside, they discover a hideout where Jonas had evidently been living. It's in disarray—furniture destroyed, supplies ripped apart, and a diary partially intact. The final entry reveals Jonas’s diabolical plan: he faked his death, preyed on his brother’s superstitions, and lured him to the tomb intending to kill him and switch places. But Jonas also wrote about something terrifying: a discovery of lower tiers of tunnels dug by an ancient race—possibly driven underground by Native Americans—that degenerated into inhuman creatures living in the blackest depths.

The last tier, Jonas discovered, ends in a smooth shaft that descends into unthinkable depths. He cleared debris blocking the way and unwittingly opened a passage to whatever now lurks below. Jonas heard strange noises but dismissed them—until he vanished.

While exploring the tunnels further, Conrad and O’Donnel hear something creeping up from below. They see yellow, slanted eyes rising from the darkness, and then a shape leaps at them. O’Donnel fires blindly. They flee back up the corridor, but hear similar sounds ahead. Realizing the labyrinth is crawling with these things, they race for the only other exit—Smuggler’s Point.

There, they find Jonas Kiles’s mutilated corpse. Whatever he awakened had claimed him, as Conrad had feared. As they escape through the door, they glimpse one of the monstrous things that killed him: a grey, shambling, dog-headed horror looming above the body.

As they flee into the night, Conrad gives voice to the dreadful truth: these were once men. The creatures that now crawl beneath the earth are the final, monstrous degeneration of a forgotten people—The Dwellers Under the Tomb.