I'm sorry to hear that you're struggling with this in your relationship. It's normal to experience attraction to people outside your relationship, whether it is monogamous or not. Being a good partner though means that we stick to the boundaries we have agreed in our relationship - cheating isn't the way to sustain a healthy, loving relationship.
I'm not saying it's wrong to date or be physical with people outside your relationship - that would be hypocritical - but it does need to be something that is freely agreed by all parties. By freely agreed, I mean that we can't just convince someone to say it's ok if deep down they feel totally rubbish about it. Sometimes, hard as it is, we have to part ways and be with people whose needs are more aligned to our own. Does that make sense?
A couple of things to think about: Is polyamory something you think you're interested in generally? Or is it that you would like to have more kinds of sexual experience before settling into a committed relationship? Both? If you want non-monogamy, what does that look like for you?
It sounds like this might be a good time for you to take a moment to think about what you want from a relationship in general. Take a look at this for starters: Supermodel: Creating & Nurturing Your Own Best Relationship Models
. Anything in there catch your eye?
I'm also going to leave this for you to look at: A First Polyamory Guide
. It doesn't sounds like your current partner is into this just now, but it might help you think about the kinds of relationship models you ideally want a bit more. Make sense?