Studies have shown

Imagine there are all these researchers running around observing & measuring different aspects of the world.

If they are systematically collecting data to explore an issue or answer a question, then they will often publish their data, analysis, and conclusion as a study.

For example, here are some researchers describing their study into the effects of social isolation on our brains:

We wanted to look more closely at how social isolation affects grey matter – brain regions in the outer layer of the brain, consisting of neurons. We, therefore, investigated data from nearly 500,000 people from the UK Biobank, with a mean age of 57. People were classified as socially isolated if they were living alone, had social contact less than monthly and participated in social activities less than weekly.

Our study also included neuroimaging (MRI) data from approximately 32,000 people. This showed that socially isolated people had poorer cognition, including in memory and reaction time, and lower volume of grey matter in many parts of the brain.

"Socially isolated people have differently wired brains and poorer cognition"(2022)

We wanted to look more closely at how social isolation affects grey matter – brain regions in the outer layer of the brain, consisting of neurons. We, therefore, investigated data from nearly 500,000 people from the UK Biobank, with a mean age of 57. People were classified as socially isolated if they were living alone, had social contact less than monthly and participated in social activities less than weekly.

Our study also included neuroimaging (MRI) data from approximately 32,000 people. This showed that socially isolated people had poorer cognition, including in memory and reaction time, and lower volume of grey matter in many parts of the brain.

"Socially isolated people have differently wired brains and poorer cognition"(2022)

We wanted to look more closely at how social isolation affects grey matter – brain regions in the outer layer of the brain, consisting of neurons. We, therefore, investigated data from nearly 500,000 people from the UK Biobank, with a mean age of 57. People were classified as socially isolated if they were living alone, had social contact less than monthly and participated in social activities less than weekly.

Our study also included neuroimaging (MRI) data from approximately 32,000 people. This showed that socially isolated people had poorer cognition, including in memory and reaction time, and lower volume of grey matter in many parts of the brain.

"Socially isolated people have differently wired brains and poorer cognition"(2022)

We've highlighted two ways to look at this snippet.

The first lens is if you are looking for the method of the study: what did they measure?

The second lens is if you are looking at how this snippet works in terms of argument components:

The issue is how social isolation affects brain matter.
The evidence is their MRI study of 32,000 people.
The claim is that social isolation reduces brain matter.
And the segment that we highlighted as the method functions as a kind of limit: it qualifies the evidence so it's more clear who is in that set of 32,000 people.

Many arguments in the news & media cite scientific studies as evidence.

Phrases like "studies have shown" are powerful because they suggest that something has been proven to be true.

For example, this article uses studies as evidence that living near water is good for your health:

There has been much research into using therapeutic landscapes in cities to bring the benefits of being in nature to more people.

But while plenty of studies have focused on green spaces, researchers are also beginning to look at the health benefits of living near water. So far, studies show that people living near water have a lower risk of premature death, a lower risk of obesity, and generally report better mental health and wellbeing. These blue spaces also reduce the gap between less and more affluent areas in the risk of dying prematurely.

"Living near water can be beneficial to your mental health"(2021)

This essay uses studies as evidence that kids might be better than adults at learning:

In fact, both neuroscience and computer science suggest that there’s an intrinsic trade-off between exploration and exploitation. Grown-up ‘executive function’ abilities such as inhibition, long-term planning and focus are great for exploitation, but might make you less likely to explore broadly. Computer scientists suggest that the best way to resolve this trade-off is to explore first and exploit later. Childhood in general, and human childhood in particular, seem to be evolution’s way of implementing that strategy. Rather than approximating the sophisticated ways that adults acquire knowledge, new studies show that children can actually be better at exploring and learning than adults.

"Vulnerable yet vital"(2020)

This essay uses studies as evidence that deliberately "bad" punctuation may not reflect poor language skills:

Punctuation is dead – or is it? If you’ve ever texted ‘im here’ or ‘its in the car’, you’re in good company. Most of us have, at some point since the dawn of texting, transgressed the boundaries of good grammar, and swallowed one apostrophe or another in the name of speed or convenience. Studies have shown that such textisms as deliberate spelling mistakes, abbreviations and omission of apostrophes don’t deteriorate language skills, but boost them – provided such texting goes hand in hand with ‘proper’ grammar education.

"Beside the point? Punctuation is dead, long live punctuation."

But is a study "proof" of anything?

If a study collects evidence and then uses reasoning to reach a conclusion (i.e. a claim)... then, a study is just an argument.

But what's special about a study is that a study's claim can be used as evidence in someone else's argument.

For example, the neuroscience researchers in the first snippet above use the claims from previous studies as evidence to support their own argument about the impacts of isolation:

Why do we get a buzz from being in large groups at festivals, jubilees and other public events? According to the social brain hypothesis, it’s because the human brain specifically evolved to support social interactions. Studies have shown that belonging to a group can lead to improved wellbeing and increased satisfaction with life.

Unfortunately though, many people are lonely or socially isolated. And if the human brain really did evolve for social interaction, we should expect this to affect it significantly. Our recent study, published in Neurology, shows that social isolation is linked to changes in brain structure and cognition – the mental process of acquiring knowledge – it even carries an increased risk of dementia in older adults.

"Socially isolated people have differently wired brains and poorer cognition"(2022)

Building on studies is a good thing!

When a study is well-designed and well-reasoned, it creates a little brick of evidence that other people can use when building their own arguments.

In that way, studies let us build on each other's knowledge over time, however incrementally.

For example, this snippet is from a proposal that builds on earlier studies about students struggling to learn physics:

Students’ “epistemological” beliefs — their views about the nature of knowledge and learning — affect how they approach physics courses. For instance, a student who believes physics knowledge to consist primarily of disconnected facts and formulas will study differently from a student who views physics as an interconnected web of concepts. Unfortunately, previous studies show that physics courses, even ones that help students learn concepts particularly well, generally do not lead to significant changes in students’ epistemological beliefs. This paper discusses instructional practices and curricular elements, suitable for both college and high school, that helped students develop substantially more sophisticated beliefs about knowledge and learning, as measured by the Maryland Physics Expectations Survey and by the Epistemological Beliefs Assessment for Physical Science.

"Helping physics students learn how to learn"(2000)

Note how, in this instance, the evidence is being used to substantiate the issue. (This piece of evidence is helping prove that there is a problem to be solved.)

(Also, the term "epistemology" (as in "epistemological beliefs") is really important when it comes to evaluating evidence and figuring out how we know what is and isn't true.)

However, there are two big problems with studies:

Studies are not always good, but people still cite them as evidence.
The conclusions of studies, even good ones, are not always accurately represented by people using them in arguments

These two problems can lead to people citing evidence that sounds convincing but is actually false or misleading.

For example, in 2008, in another example of food brands behaving badly, Dannon Corporation, the makers of Activia yoghurt, ran ads saying one daily serving of Activia was "clinically proven" to help with something they called "irregularity" or "slow intestinal transit":

From the FTC's exhibits (Source)

The U.S. Federal Trade Commission found that the cited studies did not actually support Dannon's claims about Activia and other Dannon products:

In truth and in fact, eating one serving of Activia daily is not clinically proven to relieve temporary irregularity and help with slow intestinal transit time. For example, as described in Paragraphs 12 and 13, the two Meance et al. studies utilized placebo groups but that information was withheld from the scientific journal to which the studies were submitted for publication, thereby concealing the fact that there was no statistically significant difference in transit time between the active and placebo groups, and eight of ten scientific studies conducted on Activia showed no statistically significant effect of Activia on transit time when compared to a placebo. Therefore, the representation set forth in Paragraph 17 was, and is, false or misleading.

DECISION AND ORDER IN THE MATTER OF THE DANNON COMPANY 082 3158 C-4313(2011)

The company settled for $21 million and changed its advertising to make it more consistent with the research findings, including revising the required intake of Activia from one serving to three servings daily, if you wanted help with "irregularity":

(Unlike the previous ad, the FTC thought this ad was 👍.) (Source)

And that's all just trying to agree on what studies say about yoghurt!

But phrases like "studies have shown" convey a sense of authority and consensus that can be persuasive, which means a lot of people want to use them in their arguments.

This snippet is about a study about studies, and shows how frequently they are misrepresented:

One team of researchers recently attempted to figure out how common this type of misrepresentation is in news stories and on social media.

They identified the fifty research studies shared most often on Facebook and Twitter about how factors such as diet, pollution, exercise, and medical treatment were correlated with health or illness.

Because it is very difficult to demonstrate causality in a medical study, only fifteen of the fifty studies did a decent job of demonstrating cause-and-effect relationships. Of these, only two met the highest standards for doing so. The rest identified only correlations.

That’s okay; identifying correlations can generate important hypotheses, among other things. The problem is how the results were described.

In a third of the studies, the medical journal articles themselves suggested causation in the absence of adequate evidence. Matters got worse in the popular press. Nearly half of news articles describing the studies made unwarranted claims about causation.

When reading articles about medical trials or any other studies that purport to demonstrate causality, you can’t count on the story being presented correctly. You need to be able to see through the bullshit.

Calling Bullshit(2020)

Health and life sciences have loads of great studies—that's why so much of modern medicine and biosciences works so well.

But it's also a domain in which studies can be very complex (because living systems have so many interacting parts) and there is a lot of incentive to publicise & potentially misrepresent findings (because people care about health).

Let's cite some fake studies as evidence. Here's an ad for radium emanation water (a real product, which was like the radioactive Activia of its day).

The claim is that it relieves diseases of "faulty elimination".

Make up some fake "studies show" type evidence to back it up:

Refer to 2-3 make-believe studies as evidence to support the claim in this ad.

Here's an example of some made-up scientific studies, cited as evidence:

Studies have shown that drinking Radium Emanation Water reduces complaints related to uric acid conditions like gout and lack of vigor by 40%.

This astounding result was corroborated by further research which found that subjects reported a massive decrease in frequency and intensity of headaches after only two weeks of regular Radium Emanation Water consumption.

Finally, the National Bureau of Fictitious Health Claims confirmed that Radium Emanation Water contributed to a remarkable improvement in overall vitality and energy levels due to the '"invigorating properties" of the low-level radioactivity.

In a nutshell

When scientists systematically collect and analyse observational evidence, they will often publish their findings in a study.
The claims of one study can become the evidence of another study.
This is good: it's how we build upon knowledge.
But there are risks.
Studies can be faulty. (We often see scientific papers retracted when they are found to have problems.)
Studies can be misrepresented, particularly in media and online.
However, phrases like "studies have shown" and "research suggests" are persuasive because they give the impression of consensus & authority, so lots of people use studies as evidence when they can.

Forms of evidence

Studies have shown