To some, 'shadowy figure' might be very dark, as night.
I prefer the description 'shadowy male figure' or shadowy female figure...but what you will read below is actually silly, not frightening. Maybe one of the suggestions might help?
When I was very young, a ghostly figure that glowed or was of pale illumination actually was more scary--read, I felt guilty-- to me. When I was young, if I thought in a dream that the sihouette looked like my mother, I got a sneaky feeling of 'being caught' doing something silly, like painting on walls or leaving junk around. That meant I'd have to clean my mess up or go to bed soon, which was boring. So very young, the most scary silhouette was a thin woman whose shape was bulky with her towels on her head and her body, just fresh from a shower, asking me what was I doing up at night. The towels were a light color, her shadow was dark.
Fears of an octopus-like creature, or drowning associations and having spiders on my pillow when very young got less scary as I began to learn more about ocean life, insects and plants---I tried to use fearful symbols in dreams as exploration and as a learning point, little by little.
When I was older, on my own, once I might have seen a pale face at my first floor apartment window when waking up from a dream---hard to tell, as I was half asleep. That was when my first guy told me of a Peeping Tom in the neighborhood, so I closed my shutters afterwards.
In all cases, when I associative dreams about things that I don't like or have to figure out, it is usually a pale masculine figure. I'm actually trying to paint lighter figures now, so they have better associations...I actually become comforted to see shadowy silhouettes, as they usually form the shape of my grandfathers standing upright. I have the very friendly, relaxed feeling...remember though, we all have different associations with different shapes and sizes and silhouettes.