It sounds like you have some good self-awareness of the huge amount of anxiety that sex and the risk of pregnancy are causing you. You've slowed down on having sex as often, which is a great start. I think taking a (short) break from intercourse while you find some ways to deal with this anxiety might help your mental health.
I have a few suggestions that I'd like to get your thoughts on.1. Understanding the anxiety
Heather has written about this in the past, in response to another user that also wondered why they couldn't stop feeling anxious about pregnancy. Check out this article, and consider the questions that Heather is asking, to see if you can identify where this fear is coming from. Why can't I stop being so scared of pregnancy?2. Address the anxiety with support
The intense anxiety you're feeling is something that a mental health professional can help you with. Talking with a therapist or counselor is confidential, and they'll be able to offer you a safe, non-judgmental space to discuss the guilt you're feeling about sex + the anxiety you have around sex and pregnancy. Part of working through the anxiety is not only understanding where it comes from, but also learning some general techniques for calming yourself down. This page has some resources about anxiety and mental health. What do you think about seeking some professional support? Anxiety and Other Mental Health Resources
This article also has some concrete steps you can take to deal with the shame you're feeling about having sex outside of marriage - a shame that is NOT because of anything you're doing wrong, but is from other people pushing it on you. Undoing Sexual Shame
And one more about shame, religion, and sex: How can I stop feeling so guilty?3. Find sexual activities that you can enjoy without adding pregnancy anxiety
There's so much you can do with your partner that has a very low risk - or none - of pregnancy. With all the safety measures/birth control you've been taking it sounds like your risk is already pretty low, but that doesn't seem to be helping you, so maybe removing that risk all together could help you feel better.
Incorporating some different activities into your sex life that DON'T give you anxiety is important because you deserve to have sex that is enjoyable and relaxing! Here's a list of activities with the risk level associated with each one. Can I Get Pregnant, or Get or Pass On an STI From That?
I'm guessing your partner is probably aware of your anxiety, so you could present the idea of taking a break from higher-risk activities in a really positive way. "I want to enjoy sex with you but my anxiety is making it really difficult. I'm working on solving it, but for the meantime, could we switch to some other fun activities so that this can be a thing I enjoy without the terrible anxiety?"
What do you think about those three steps, as a start?