I love it when a plan comes together

"I love it when a plan comes together" John 'Hannibal' Smith, The A-Team
Another dodgy piece of news came across my desk this morning and I was struck by the fact that it seemed to be a nice coalescing of the topics of two of my recent pieces - one on the unreliability of food questionnaires in nutritional research and the other about the follies of a vegetarian diet. As I digested the connection between the articles, the catch phrase of John "Hannibal" Smith from the A-Team (image above) immediately came to my mind.

The piece in question, from Newsweek, is headlined "Eating Salad Every Day Keeps Brains 11 Years Younger and Prevents Dementia, Study Shows. 'Wow!', I thought, 'that's great! I eat salad all the time, so I must have the brain of an infant!' But as is usually the case with headlines like this, it's actually not true. Or, at least, it's not shown to be true, despite the catch-all term "study" in the title.

Citing a "study" is like saying Jesus himself came down from heaven and declared something as incontrovertible, undeniable truth. To say a study shows something means that the gods of science have this figured out. If you're 45 and you eat salad, your brain is only 34 years old! That's science!

spring green salad

Exhibit A: "spring green salad"
Quoting from Newsweek:
Nutritional epidemiologist Martha Clare Morris and her team at Rush University Medical Center in Chicago found that people who ate one to two servings of leafy green vegetables each day experienced fewer memory problems and cognitive decline compared to people who rarely ate spinach. In fact, Morris estimates that veggie lovers who included about 1.3 servings a day into their diets had brains that were roughly 11 years younger compared to those who consumed the least amount of foods like spinach or kale.
Sounds like an impressive finding, even if it does conflate all leafy green vegetables with spinach. And how did the crack scientists come to this amazing conclusion? Perhaps it was via tightly-controlled salad feedings, compared to a placebo group (yes, placebo salad. It's probably a thing), and subsequent measuring of dementia markers?

Not quite. They gave out food questionnaires, assumed people were not only entirely honest in their answers, but had perfect recall of what they ate over the last who-knows-how-long and then crunched the data (don't get me started on food frequency questionnaires). This is not a randomized trial, where researchers adjust a variable of some kind and see what happens. If it were, researchers would be able to say for certain that the variable introduced was the cause of a change they observed.

chopped salad

Exhibit B: "Mediterranean chopped salad"
The study in question is an 'observational study', where the researchers just observe groups of people, record what they did (or have them record it, because they're lazy) and crunch the numbers to see if anything one of these particular groups did is correlated with any outcome (usually disease or health markers). An example would be looking to see if there's a correlation for ice-cream consumption and contracting seal finger. In this case, the researchers would divide the group into those who ate ice-cream, how much of it they ate, and whether any of them developed seal finger. Then they'd analyze the data to see how much correlation there is between the consumption of ice-cream and coming down with seal finger.

But these researchers didn't take a bunch of people and feed them salad to see what happened. They simply took a group of people who were recording the foods they ate (we'll just ignore from here on out how unreliable that is) and analyzed how they performed on periodic cognitive assessments. When they went through the data, a correlation came up between salad eaters and those who did well on the cognitive assessment.

caesar salad

Exhibit C: "caesar salad"
You can't say based on that observation, however, that salad has an effect on dementia. All you can say is that there is a correlation. It ignores many of the confounding factors. Even when researchers try to control for confounding factors, they can never get them all. One that wasn't looked at in this study was sugar consumption. From the study itself, they say "In a linear mixed model adjusted for age, sex, education, participation in cognitive activities, physical activities, smoking, and seafood and alcohol consumption, consumption of green leafy vegetables was associated with slower cognitive decline." I don't see sugar on that list. But it's reasonable to assume that those who regularly eat salad also adhere to the widespread assumption of what conforms to a 'healthy diet', which would likely include eating a lot less sugar, processed foods and trans fats. Any of these could be responsible for the results the researchers found. The results don't necessarily have anything to do with salad.

Or maybe the relationship is actually reversed - having dementia actually makes you less likely to eat salad. Maybe an unknown side effect of dementia is a distaste for salad. The fact is, from this study, we don't know that this isn't the case.

Observational studies aren't necessarily bad, in and of themselves. They can be indications of what needs to be researched further through intervention trials. However, you can't give definitive statements about causes from studies like this. All you can say is that there's a correlation between two factors. So the researchers could say, in this case, that eating salad is correlated with having a "younger brain" (whatever that means). But that's not what the headline says. In fact, media headlines fail this test ALL THE TIME.

Observational researchers themselves are often quoted in press releases making the same mistake, implying their findings show causation, or saying their results can be used to dictate policy decisions, or that people should use the findings to change their behaviour (whether they're saying this because they actually believe it or because they're trying to inflate the relevancy of their study is a mystery. Not sure which is scarier). Think about that every time you see a headline saying 'eating meat gives you diabetes'.

crazy salad

Exhibit D: "crazy salad"
The article does actually admit this fact, which means either this journalist kind of knows what she's talking about, or the researcher actually emphasized this fact. In the second to last paragraph we get this:
Because this study was observational, the data provide no concrete evidence of a causal relationship between spinach and brain health. For now, the connection is only a correlation, and does not extend to younger or nonwhite or Hispanic people. But adding a serving of spinach into your diet is never a bad idea.
That last sentence is what gets me, though. You might as well act on this advice, even though there's "no concrete evidence" that it's true, but just in case. The amazing thing about this media report on a scientific study is that the body of the text directly contradicts the headline (which is all most people ever read), but the sloppy journalism and questionable ethics is justified by telling the reader that it's good advice regardless. I'm sure the Newsweek staff are having no trouble sleeping tonight, thinking, 'If I just get one more person eating salad, it all will have been worth it.' If I was conspiratorially-minded, I would be thinking this study got funded by the salad industry. Big Salad's oily and vinegary tentacles are everywhere.

spinach salad

Exhibit E: "spinach salad"
The state of nutritional science is pretty terrible. From 'Self-destruction of science: Most findings are wrong or useless':
And then there is the huge problem of epidemiology, which manufactures false positives by the hundreds of thousands. In the last decade of the 20th century, some 80,000 observational studies were published, but the numbers more than tripled to nearly 264,000 between 2001 and 2011. S. Stanley Young of the U.S. National Institute of Statistical Sciences has estimated that only 5 to 10 percent of those observational studies can be replicated. "Within a culture that pressures scientists to produce rather than discover, the outcome is a biased and impoverished science in which most published results are either unconfirmed genuine discoveries or unchallenged fallacies," four British neuroscientists bleakly concluded in a 2014 editorial for the journal AIMS Neuroscience.
Unchallenged fallacies like, say, the erroneous belief that someone with Alzheimer's committed the mortal sin of not eating enough salad? 'Related to' studies are pretty much useless in the grand scheme of things. They may be pointing to a real relationship, or they may be pointing to nothing. But as long as the media keeps reporting on them as if they're fact, the public mind gets increasingly filled with lies and half-truths and we get more and more confused about what we actually should be eating. In an era where media companies make a significant portion of their income via 'clickbait', sensationalist headlines are literally gold. If telling people salad is the cure for Alzheimer's makes Newsweek more money, because science, then that's what they're gonna print.

tossed salad

Exhibit F: "tossed salad". Note utensils for "tossing" (right).
For a look at some actually helpful advice regarding Alzheimer's and dementia, see Dr. Gabriela Segura's article 'The health program for Alzheimer's disease that mainstream treatment fails to surpass'.