Thank you for those additional details, they're quite helpful! To make sure I'm following right, it sounds like you do experience some desire for sex, just not the amount you assume you're supposed to? And when you've been sexual with him, you've felt some level or turned on, or not at all?
It might help to read this article, since it does a good job of outlining what it is to asexual: Just the Basics, Ace: An Asexuality Primer
. That being said, it sounds like you do experience sexual attraction of a kind. And, more specifically, that you feel attraction to more than one gender. That certainly matches the definition of bisexuality and pansexuality. That being said, you get to the be the ultimate decider of what you label your sexual orientation (or if you choose to label it at all). When you think about being bi or pan, how does it make you feel.
I'm wondering if it might help with those feelings of confusion to give this article a read: Sexuality: WTF Is It, Anyway?
. When you're going through that, are there certain circles of sexuality that jump out at you as matching how you feel?