How Do We Science?
Science was my favourite subject in school. I found maths repetitive and English…confusing: there were always a hundred ways to look at a story, none of them necessarily right or wrong. My friends reveled in that fluidity, but I always ended up just feeling lost. Science, though: science gave me a structure I could use to explore ideas about the world.
Some people aren’t so sure about science, or don’t get what it’s all about and why we really can feel confident about a thing when it’s come from science. We see this show up with some folks at Scarleteen who struggle to be sure of things like birth control efficacy rates and how human reproduction actually happens, or have a lack of confidence that condoms do exactly what scientific study has found they do.
Sometimes users don't trust any of that, but will believe their cousin who tells them something did or didn't work for them personally, instead. What’s the difference between scientific knowledge and the lived experiences of people we know? Well one tells you something very individual and specific – about a person and their experience – and the other is much more general so it can be used reliably, and for many people, to make decisions based on what’s likely to happen in a given situation.
I think that what science comes down to really, is this: it’s simply a sound structure we can use to ask questions, and then find out if we’re getting closer to the right answer, or at least can always and reliably use to rule out the wrong ones.
That structure is a set of steps, called the scientific method.
Put simply, those steps are:
- Ask a question
- Suggest what the answers might be
- Test those answers
- Choose the best answer based on those tests
That’s it. Really. When someone talks about science as if they’re a member of some super-selective cult, all they’re really saying is that they like to test suggestions before accepting that they are true.
Of course, there’s plenty of intimidating jargon to go with the process. I’ll try to bust that in the rest of this piece by using an imagined experiment to put them in context so you can understand what they mean in the real world. If all goes well, you’ll come out with a good understanding of what the scientific process looks like and why science is so utterly great. If it goes really well, maybe you’ll start planning experiments yourself. Either way, I hope you’ll bear with me if I get too carried away and we get technical!
1. Asking the question
What do you want to know about the world? Get specific. Maybe my big question is, “How can we make the world a fairer place?” Good question, me, ambitious! A bit too ambitious, actually. To answer something like that we’d need to sort out a whole bunch of much smaller questions, like about the kinds of inequality that exist and what we can do about them.
One inequality inside all the unfairness on the world is that the people who’ve got a uterus (who are usually, but not always, women) disproportionately bear most or all of the burden of birth control. So, let’s start with this: “What’s an effective reversible birth control for people with testes?”
2. Suggest answers
Now we can come up with some ideas, for example:
- A hormone gel*
- Special gunk injected into the sperm ducts
- An ultrasound of the testes
These answers can be turned into simple statements that can be proven wrong. This makes it possible to test them; we call these HYPOTHESES (which rather remarkably rhymes with testes). Let’s look at how we’d test hypothesis A.
Hypothesis: That a cream containing progesterone and testosterone, at a specific dose, applied to the shoulders and upper arms of the partner identified as male daily will significantly reduce the chances of pregnancy within fertile couples.
We also will have a NULL HYPOTHESIS which basically says that the thing we are testing makes no difference. In this case:
Null hypothesis: That a cream containing progesterone and testosterone, at a specific dose, applied to the shoulders and upper arms of the partner identified as male every day, will NOT significantly reduce the chances of pregnancy within fertile couples.
3. Test those answers
Before anything else, any study like this needs to first pass an ethics panel. Since we’re testing on humans, we’re also going to need to make sure that everyone knows what they’re signing up for and is freely choosing to do so. This is INFORMED CONSENT and it's really important. Like sexual consent, it’s tied up in the principle of bodily autonomy – my body, my rules. Historically, many doctors abused their positions of power to conduct experiments on marginalised communities without their knowledge or consent. Both ethics panels and clear, informed consent are crucial to protecting the rights of people participating in studies.
Next, we’d want to advertise widely so we can get a whole list of people volunteering to take part – the broader and more diverse our group the better, so that we know how this works for all couples, not just, y’know, white college students. To make the experiment fair, we'd need to ask them a list of questions to make sure they’re eligible – are they a couple who could co-create a pregnancy without outside intervention? Are they having penis-in-vagina intercourse on the regular? Are both partners happy to not use any other forms of birth control for a year? Are both partners okay with a small but unknown risk of pregnancy? Questions like these make sure that a reduction in pregnancies is due to the thing we are testing, not other factors like the person not having the kinds of sex that could result in pregnancy. This gives us our SAMPLE POPULATION – the subset of all the people with testicles in the world that we are actually working with.
In most studies, we’d then divide our sample into two groups: one will get the real treatment and another CONTROL group will get a placebo – a version of the medicine with no actual drugs in it, in this case the gel, minus the hormones. A control group is used just in case anything in the way we’re running the experiment, other than what we’re trying to measure, affects the results. If everything goes to plan with the control, we can assume the experiment works and what we see in the results is genuine, rather than, say, a dodgy piece of equipment giving wonky readings. Testing contraceptives though, involves real couples that generally do not want to get pregnant and so we’ll more commonly compare to another existing method and to what we know of pregnancy rates in the general population.
Since the best existing temporary contraception for people with testes is condoms, we’ll compare our new gel to them.
We’ll then RANDOMLY SELECT which couples in our experiment will use the new gel and which will use condoms, like by giving every couple a number and using a random number generator to allocate them to the two groups. We’ll end up with half our sample population in each group. Let’s say we have 5000 couples, so there will be 2500 in each group. That’s a pretty good SAMPLE SIZE. The bigger the sample you have, the more sure you can be that the effect is because of the thing you are testing, not just dumb luck – just trying out the gel on one or two people wouldn’t be enough.
Large sample sizes are what makes something learnt through scientific process more relevant when making a decision than a story from a friend about someone they know. Yes, your friend is telling you about a real person, one you could probably talk to if you wanted. But one story, all by itself, doesn’t give a complete picture of the diversity of possibility, or tell you how likely something is. In a big sample, that one story is represented, but so are hundreds of others.
Our couples can then go off and use the contraception gel according to our detailed directions. They’ll be keeping diaries of the intercourse they have and reporting any pregnancies over the course of a pre-defined length of time: let’s say a year. At the end of the year, we can then compare the two groups to each other, and to what we know of pregnancy rates without any contraception.
4. Choose the best answer based on our tests
Without any contraception, we’d expect that within couples having the kinds of sex that create pregnancy, about 2125 of our 2500 couples would conceive in the year of our study. Say that at the end of our study, 350 couples using condoms have become pregnant (based on “typical use” effectiveness rates), and 225 couples using the gel have. Statistical analysis (the number-crunching at the end of an experiment to help interpret the results) is a subject for another day, but suffice to say that 225 is a whole lot less than the 2125 pregnancies we might have expected without the contraceptive.
If and where the evidence is against a hypothesis, that hypotheses is rejected — because we then know that hypotheses was wrong! If we can’t prove a hypothesis false, no matter how hard we try, we can safely assume that’s because it’s true. Looking at these results, and then back at our hypothesis and at our null hypothesis, it seems we can safely reject our null hypothesis that the gel does not significantly reduce chances of pregnancy, based on the evidence from our trial. We will therefore provisionally accept our other hypothesis that it is an effective form of contraception, meaning we will go ahead and make decisions on the basis that the gel works, unless new evidence comes along proving us wrong.
By the way: Often, we'll hear the word “theory” used about a scientific idea. In science, theory doesn’t mean a hypothesis that we’ve just thought up to test a relatively straightforward question. Usually, a scientific theory is an answer to a much bigger question, one not just supported by evidence from one experiment, but by huge amounts of information gathered from loads of experiments over a long time. When we talk about a scientific theory, we’re talking about something we’re actually really very confident about.
Think about science as having multiple parts: there is the scientific method – the structure we’re working through in this piece – and scientific knowledge – all the things we learn by going through this process to test and refine our hypotheses.
This leads to two of the key things about scientific knowledge; that it is true no matter how we feel about it, and a good scientist is not so attached to a theory that they won’t throw it in the bin as soon as there is good evidence to disprove it.
For example, whether I believe in or like the Theory of Gravity, I will not suddenly start floating up and away from the earth. Newton first conducted his experiments on gravity over 300 years ago. His theory worked and was widely accepted until Einstein revolutionised physics with his Theory of General Relativity 200 years later, showing up flaws in Newton’s version along the way. When new evidence came along, scientists accepted that the theory they were working with needed updating and did so.
Once we’ve gathered evidence and come to a conclusion, we’ll write up our study so that others can both learn from and try to replicate it. Our report will include not only our results and conclusions but also our METHODOLOGY – that’s a clear and complete description of how we went about conducting our experiment. This should read like a set of instructions, so that with the right equipment anyone can do it and check our results.
When a paper is reviewed by other scientists not involved in the study prior to publishing (called PEER REVIEW), the methodology will be looked at closely to make sure that it was sound and the conclusions logically follow from the results. Only after a paper has passed peer review can it be reliably used to make claims about the safety of a medicine or the effectiveness of a birth control method. After peer review, the paper can be published in a reputable scientific journal, and it’s at this point you are likely to hear about it in the media, as it gets attention outside the scientific community.
What you’re less likely to see on the news is that after the first paper on a new idea comes out, that idea will be tested in new ways by different teams to constantly refine and build on it. Studies of new contraceptives on thousands of people are only possible once the drug has been through years of development before being tested on the first person, and then tried out on a tens and then hundreds of people. One study all by itself doesn’t tell us much, but dozens of similar studies together build a good picture of how things work.
Science gives us a structure – a process – to follow to answer questions about the world, and anyone can do it. Okay, maybe you can’t get together the backing needed to do a study on thousands of people right now, but anyone can do some kind of science. It’s about stepping back, and instead of listening to one voice or a few experiences, bringing all the evidence you can find to the table. It’s about testing your ideas and making them stronger; about finding what doesn’t work, learning from that and trying again.
It can be hard to have faith in science if you’ve grown up around a lot of anti-science thinking. But that’s okay, because science isn’t about faith, or what we believe: it’s about what we can prove. Knowing something through science comes from looking outwards at our world in a careful way, rather than looking inwards to ourselves, how we feel, or how we react to stories we hear.
This is where understanding the scientific method is a formidable tool – knowledge is power, and when you know what good science looks like, you have the power to make much more informed decisions about your future, and based on the best possible information AND your own feelings. You might be making a decision about your body, or even about the future of your world when you take this knowledge to the polls. So go out there, ask questions, weigh up the evidence and make that knowledge work for you.
* These methods are all really being investigated right now! The hormone gel is in early trials, using a smaller number of people. If it goes well, showing evidence of effectiveness without particularly bad side effects, studies will expand to include more couples. The others are in earlier stages of development and may or may not progress – either way we’re looking at about 10 years for any of them to be fully on the market.