MONSTRUM

THE BEAST AND ITS SIGNS
The word "monster" comes from the latin monstrum where it means both a thing which is not of the natural order but also a sign, omen or portent of the same. It is both the thing in the dark, but also the claw marks it leaves on the door, the way animals walk backward when it is near, an eclipse heralding its birth. 

Also, we've got internal/safe spaces and external/potentially dangerous spaces. Horror has monsters lurking on the outside, trying to get into the safe spaces or it has a revelation that there is no internal/safe space, that the whole world is monstrous (or it has some combination of the two). In both cases, we're talking about ontological rupture of the first order, of the truest essence: something that makes being human as we understand "human" deeply problematic if not impossible (or at least, I feel like good horror should do that).

YOU CAST YOUR EYES BEYOND THE VEIL, ON THE HORIZON... SOMETHING
Where you put magic-users is up to you, but:

1. When you make a Magic-User, tell the GM what is out beyond the veil, chasing you, trying to get in. You might be wrong, you might have seen it incorrectly. It might have changed its form.

2. Whenever you cast a spell, roll a d20 and tell the GM the result. The GM keeps a record of the results. It's always getting closer.

THEY ARE EVERYWHERE, THEY ARE HIDING
D&D provides two approaches to this. On the one hand, you've got points of light in a howling wasteland populated by the monstrous (Greyhawk, the encounter tables in the first edition of D&D and the map it suggest you use for your game world all are of this order); on the other hand, you've got relatively civilized, late medieval peoples living their lives and then tucked away in a few dark corners is an alternate world of monsters (B/X and most of the Basic modules have this more Beowulf feel to them). The former seems to often be the end result of a campaign, the latter its starting point.

Both can work as a kind of horror game. Where D&D really falls down on the horror front is the monotony of dungeons and the way it deploys monsters.

EXCERPT
A part of the solution that I've found has to do with how you talk about things. Here's an excerpt from something on which I'm working:

"DENY CLOSURE
Give Players however much information they want; their characters still have to dig around for it, they'll still have to spend Turns and actions poking around into thing, but they get the information. They get information, but no explanations:

they are in a dark tunnel and they see eyes in the dark, reflecting like a dog's, and they hear oncoming steps, quick breathing, the sound of metal hitting stone.

one of the magic-users throws a torch forward and they see something human shaped, clothed in rags, moving towards them at a run.

two fighters move forward, jabbing into the dark with their spears.

in the aftermath, they find a body, mangled.

Players: What was it?

You: Do you want to look at it? How are you going to inspect it?

Players: We'll poke it. If it's lying face down, we roll it over.

You: (because rolling it over sounds good and because you want to be verbose) it's human-shaped but small, maybe like a teenager or a malnourished teenager. It's bony and its skin is sort of grey, but it's also very dirty so you'd need to wash it off to tell its skin color for sure. Maybe the proportions are a little off? Maybe it's legs are a little short? It's wearing rags that look like they might have been clothes once. It's not holding anything. It has stringy hair. It doesn't respond when you poke it, it's like poking a raw turkey or chicken. You roll it over with a pole, which one of you is doing that? How close are you getting?

Players: I'm rolling it over and I guess I'll put a torch down by it and get close, but I'll have a shield up, in case. I'll have my spear like, on it.

You: OK. You roll it over with a pole and its arm flops to the side. It's nose is bloody and its face looks mostly human, though its mouth looks too big, maybe? There's something weird about its mouth. Anyone want to put their hand in its mouth?

Players: Nope. So is it like a human?

You: It's like a human, sure. It looks a lot like a human."

What they've just fought, in terms of stats, is a goblin. When they find more goblins later, they all look like the first one. There are also children like this goblin, and they're just as dangerous. Some don't speak, others speak in a language no one recognizes.

It's possible that in another bolthole there are goblins that look like this first one. There are certainly other goblins that look the same but who work differently. One can only be harmed in the light. Another heals whenever it's in the dark. A third heals any injury, even fatal ones, if buried in the earth. Others make elaborate traps. Some worship something made of blood and fire and it is their king and it gives them great strength and sight beyond the veil.

That's part of the puzzle anyway. The other part has to do with mechanics.

attribution: Sam Wolfe Connelly (who is amazing)