Headaches during pregnancy are not something to be taken lightly. They can signal Pregnancy Induced Hypertension (among a host of other complications) which puts the mother at risk for delivering an unhealthy baby.
When I was pregnant almost 2 years ago, here's what my OB said. Tylenol is safe to take during pregnancy for minor aches and pains, but ONLY Tylenol, and ONLY in small doses (1 to 2 tablets). No Ibuprofin, no Asprin, no Aleve.
The reason many medications are off-limits during pregnancy is they have been shown to cause birth defects, stillbirths, and miscarriages in lab rats. The placenta does not filter out these or any other sort of drug, and when you take medications, your baby gets a strong dose of the medication too.
If you aren't already seeing a doctor on a monthly/bi-monthly (depending on how far along you are) basis for prenatal care, you need to start TODAY. I cannot stress enough how important this is.
Tell your doctor about your headaches, and describe exactly what they feel like. Keep a log of how they feel and how often they occur to share with your doctor at your next prenatal visit.