Got it. Since you two haven't been able to use condoms, are you also making sure you're up to date on your STI testing?
Okay, so it sounds like you've had bad experience with one method of contraception in the past, which can definitely suck. When you decided to go off that method, did you and your healthcare provider talk about possible alternatives, ones that may not cause the same side effects or ones that are not hormone based, such as
Cervical Barriers (Diaphragms, Lea's Shield and Cervical Caps) ? Too, it can help to remember that while weight gain is a possible side effect of some birth control, it's basically a guaranteed outcome of getting pregnant ( I don't say that to be snarky, but rather because sometimes we can get so caught up in worrying about the side effects of birth control that we forget the variety of effects pregnancy has on the body).
In case the information is useful, withdrawal is 73-82% effective with typical use over a year. If you haven't already done so, you and your partner may want to talk about whether that is a level of risk you're both comfortable having.