Hey Learningthings -- based on your description, it sounds like these bumps are ingrown hairs. These occur when a shaved or tweezed hair grows back but directly into the skin, often causing inflammation in the form of small, pop-able bumps. Generally, these do go away on their own but if you're finding that they're lingering for weeks or start to hurt, I recommend talking with a healthcare provider for additional treatment.
Also, for clarification, when you say they're not on your vaginal lips -- do you mean they're not on your outer labia? Can you help us understand which area you're referring to? For reference,
here's a diagram.