What you're describing is actually totally common! If you're someone who experiences sexual attraction and you're around someone you're attracted to, it makes total sense that you'd get aroused or think about sex. That's especially true when a relationship is fairly new, because there's an added element of excitement from "oh my gosh, I get to be near this person." And when you're into someone, even small things like hearing their voice can be a turn on.
That being said, there can definitely be times where you want or need to focus on other things besides how horny you feel. When that happens, one thing that can help is to acknowledge that the feeling is happening, and then shift your focus to something else. If you have the privacy to do so, masturbation can also help with releasing some of that pent-up arousal.
With those fears around what he'd think if he found out about your desires, has he said or done anything to suggest he thinks you wanting sex is a bad thing?