Welcome to the boards! This sounds like it's been tough.
I know I'm pretty sensitive to these things too, and even if I know someone is simply expressing anger in very minor (non-smashy) ways, it still scares me, and is deeply unpleasant. They might be letting off steam in a socially acceptable way, but it still feels like all that steam just goes into me, and it hurts.
I try to be more accepting of small things like being cold, or storming off, but punching and smashing things, in my vicinity is a level up and it hurts me, even if it isn't intended to, and even if I'm not the the object being hit, I can't take it.
I wouldn't treat this as a phobia or irrational anxiety because lashing out to your surroundings is something that aggressive people do and to feel fearful is a very natural response.
I've worked with lovely children who lash out really aggressively when they are upset and I have had to treat the safety concerns as something that has to be handled and don't feel like it takes away from my care for them to acknowledge that I'm not going to put them alone in a room with someone who is likely to be hurt either directly or indirectly by an outburst.
I don't think it's super different with adults. It sounds like your boyfriends behaviour is something that would make most people anxious and stressed and you don't have to be exposed to that behaviour just because you care about him. Different ways to manage anger and frustration are learnable, which don't make people feel the way you're feeling.
Do you think you could ask him to look into anger therapy or anything like that?
"In between two tall mountains there's a place they call lonesome.
Don't see why they call it lonesome.
I'm never lonesome when I go there." Connie Converse - Talkin' Like You