If it helps to know, there's nothing inherently wrong with being curious about being with other partners or wanting to explore your sexuality. The tricky bit, as you're discovering, is when that desire to explore comes into conflict with current relationships. I think you're right to be cautious about the ways in which experimenting with other partners may hurt him or them. Opening a relationship takes discussion and planning, and lots of queer women have had some not-so-great experiences being the person someone experiments with (not that you'd do that, simply that this is something that happens).
It may also help to parse out for yourself what you mean or want when you think about exploring your sexuality. Is it only to have sex with women? Or do things like trying out different types of fantasies during masturbation or consuming sexual media about sex between women also appeal to you? When you think about going on dates with women, are there things you assume will be different from dating men?