The story begins with James Conrad explaining to Professor John Kirowan his fascination with the tragic life of Justin Geoffrey. Geoffrey, a mad poet who died raving in an asylum, is of particular interest because his descent into madness appears entirely unique within his otherwise unimaginative and stable family. Conrad, after detailed genealogical research, is convinced Geoffrey's insanity was acquired, not inherited.

The turning point in Geoffrey’s life came at the age of ten, when, after a brief disappearance near a mysterious old house by the Catskills, he began to suffer terrifying and recurring dreams that shaped the course of his life. The dreams marked the start of his withdrawal from others, his nocturnal wandering, and his obsession with macabre and bizarre subjects that later infused his poetry.

Despite his family's disapproval and attempts to suppress his creative tendencies, Geoffrey pursued knowledge of ancient languages and magic rather than formal education. He eventually moved to Greenwich Village and gained some recognition as a poet before dying at the age of twenty-one.

Conrad traces Geoffrey’s transformation back to his overnight stay near the ancient and enigmatic structure known locally only as "The House." Located near Old Dutchtown and surrounded by ancient oaks, the house has no known history, and its ownership is disputed. Even the mayor of the town is unable to shed light on its origins or the reason for its eerie aura. Conrad shows Kirowan a photograph of the house, which Kirowan recognizes from a painting by their artist friend, Humphrey Skuyler.

Conrad, Kirowan, and Skuyler visit the artist’s studio, where Skuyler describes the same sense of intangible dread evoked by the house. The three decide to travel together to Old Dutchtown to investigate. Upon arrival, they meet the mayor, who reiterates the house's mysterious nature: it has been uninhabited for at least fifty years, the surrounding land is infertile, and the house is sealed tightly with no apparent way to enter.

As the trio approaches the house, passing through the circle of gnarled oak trees, they all sense the aura of unnatural dread it emanates. The mayor explains that locals avoid the place and that even attempts to cut the trees surrounding it have met with accidents. Rumors and fear, though never confirmed by any overt phenomena, surround the location. The story ends as the men prepare to further investigate the sealed house—its secrets unknown and perhaps best left untouched.