Misguided Attempts to PURE-ify Nutritional Epidemiology

David L. Katz, MD, MPH, FACPM, FACP, FACLM
LinkedIn
I recently contributed a quip to my social media channels I hoped might mature into an aphorism or meme: wherever dogma barks, we have a very hard time hearing one another. Whatever this puppy matures into, it represents my conviction on the matter.
wherever dogma barks, we have a very hard time hearing one another

Noisy dogma obviating hearing reminds me of another quip, this one from my childhood, and generally directed at me by an adult (more often than not, my beleaguered mother) through gritted teeth: you are so noisy, I can’t hear myself think!

I was a clever kid, or perhaps more accurately, a wiseass- so I had a rebuttal to this one: that’s because thinking doesn’t make any noise. I had one for my messy room, too: who am I to wrestle with entropy? Yes, I know- you are very glad you aren’t my parent. Moving on.

I contend that in the domain of nutrition, dogma is barking nearly all the time- and it exacts both the toll I claim, and my mother’s as well.  We have a very hard time hearing one another and allowing for separate sides of a common reason; and we may even be drowning out our own thinking into the bargain.

Katzes dog Barli in the snow

  • Barli – who only barks for good reason, and never bites…

The deafening suppression of thought in nutrition is often expressed in the guise of patronizing tyranny, claims of the only way things must be. Three such proclamations reign supreme.

The first is that the only way to know anything is from randomized controlled trials. This has invited and abetted some very serious mischief that has threatened progress pertaining to diet for the health of people and planet alike. The contention, however, is patent nonsense. Anyone who has ever looked both ways before crossing a busy street, used an oven mitt to handle a hot pot, relied on water to douse a campfire, or admonished a child against running with scissors- or pick any of infinite alternative examples- appreciates how much we humans “know” with nary a RCT or meta-analysis to cite.

The second– and the signature misstep of the PURE investigators– is that a larger sample is always the best route to truth.

Personally, I am a fan of large samples.  In general, the larger and more diverse a population studied, the more information it provides about conditions in the real world, rather than the contrived context of some carefully curated study cohort. Other things being equal, large samples tend to be better than small for getting at the truth.

But note that prerequisite: other things being equal. This critical element derives from Epidemiology 101. With all due respect to what I presume is a well-intentioned group behind the PURE studies, they seem to have missed this course.

Their latest paper in the BMJ tells us that “The Prospective Urban Rural Epidemiology (PURE) study has the distinct advantage of examining diets from diverse populations in low, middle, and high income countries in multiple regions across the world.” The article in question reports specifically on associations between grain intake- refined, and whole grains- and cardiovascular outcomes in nearly 150,000 adults from 21 countries. Giving credit where it’s due, the research team certainly crunched their numbers for all they were worth, preforming an array of sophisticated, multivariable analyses, including meta-analysis by region.

The principal finding was that higher intake of refined grains was associated with higher rates of cardiovascular disease and mortality. In general, I am tempted to wonder if there is anyone living under a rock large enough to consider it “news” that refined grains are not very good for us? But looking past that, it might still be newsworthy to find that refined grains exert comparable, ill effects on populations as diverse as that of PURE. But that’s not what these authors found. Rather, in their own words: “When we did a meta-analysis of the PURE data by regions, the association of refined grains with clinical outcomes was apparent in China but not in other regions.

Translating that statement from research-speak into the vernacular, it basically means that while the study included 21 countries, the association between higher intake of refined grain and adverse health effects was actually found in just one, China, where it isn’t really news. China has long been undergoing an unfortunate “nutrition transition,” adopting the dubious dietary practices of the West. These include more meat, more sugar, more refined grains- and decreased intake of both whole grains and legumes. Of note, while the BMJ paper noted participants from 21 countries, nearly one third of the total cohort was contributed by just China.

Why didn’t these researchers find the same association between refined grain intake and adverse outcomes in the other 20 countries? And why didn’t they find the strong, protective effect of whole grains reported in many prior studies?

The simplest and most probable explanation is: countervailing influences varying by population. If, for example, whole grain intake is highest where poverty is most prevalent, then rates of cardiovascular disease and mortality might be high where whole grain intake is also high, but not because whole grain intake is high. Whole grain intake might, in fact, be quite protective- but not protective enough to compensate for subsistence, poor diet quality, or the duress of indigence. Such critically important “co-variables” come in two basic varieties, confounders and effect modifiers, and both can completely subvert the implied merit and apparent meaning of an observed association.

Stating it bluntly, and again with all respect due to the effort and intention of the researchers involved, the homogenization of diverse populations to evince one-size-fits-all answers in the PURE study is a fatal flaw. An analogy may best demonstrate this.

Imagine a study to examine the health effects of swimming. Imagine further an effort to capture not just the general effect of swimming, but the one, true, “pure” effect of swimming. To that end, a population is recruited to be as large and diverse as possible.

Now, some such diversity might be entirely appropriate. Do the purported health benefits of swimming pertain, for instance, to both males and females? To younger people and older? But there is a point at which excess diversity sounds the death knell of sense, along with science. In the case of our swimming study, that point is crossed on the way to the water, and swimming lessons.

So, imagine, finally, that our aim at “diversity” lands us a large group including those who do, and those who do not, know how to swim.

Testing our exposure, we would find that those who don’t know how to swim routinely drown when attempting to do so. Thus, swimming- or failed attempts at it- is quite “bad” for them. On the other hand, those of our study participants who know how to swim reliably derive the expected cardiovascular benefits, and therefore- swimming is quite “good” for them. Pooling these data in the service of large samples, diverse populations, and summary judgment, we might report that the net effect of swimming on health appears to be…nil. The deaths it prevents are matched by the deaths it causes.

I trust no more need be said to rest the prosecution of such misguided epidemiology. The reported results of such a study would be pure nonsense.

The third bit of despotism derives, then, from the first two: if we never have enough RCTs, and if our sample sizes are never large enough, then we never put a hypothesis to bed. In other words: we never know anything. This silly contention propagates a cottage industry in questioning reliable answers we already have, and diverting essential resources from pursuing the new answers we actually need- nowhere more so than in nutritional epidemiology. The pseudo-confusion this conveys to the public at large is an invitation to keep treating junk as a veritable food group while awaiting more reliable answers. The sad result is ever expanding corporate coffers for Big Food and Big Pharma alike, in tandem with ever expanding waistlines and the dominion of chronic disease.

Let’s bring this to a close with a final, bracing reality check.

The fundamental benefits of routine physical activity to human health are indisputable, predicated on a vast assembly of science diverse in its methods, confluent in its conclusions. The fundamentals of activity patterns conducive to such benefits are comparably clear. Innumerable details about the relative merits of this activity or that, done this way or that, for this long or that long, by this group or that…are, of course, currently unknown, and in some cases perhaps so hypothetical and rarefied as to be all but permanently unknowable. We accept, though, that we can both know with confidence and even certainty the generally reasonable ways to derive the generally great benefits of exercise, even as we probe the myriad specific means to the same desirous ends; even as we compare and contrast in their every particular the many variants on the same basic theme bounded by science, sense, and global expert consensus.

The fundamental benefits of eating well routinely to human health and planetary health are equally well known, predicated on a comparably vast assembly of science diverse in its methods, confluent in its conclusions- and thus ought to be comparably indisputable. The fundamentals of dietary patterns conducive to such benefits are comparably clear.  Again, innumerable details about the relative merits of this exact dietary approach versus that, these specific foods versus those, a macronutrient threshold higher or lower, my recipes or yours…are currently unknown, and in some cases perhaps so hypothetical and rarefied as to be all but permanently unknowable. Somehow, for nutrition uniquely, we allow the latter to corrupt the former and perennially disparage the confidence and even certainty we should have regarding the generally reasonable ways to derive the generally great benefits of eating well, even as we probe the myriad specific means to the same desirous ends; even as we compare and contrast in their every particular the many variants on the same basic theme bounded by science, sense, and global expert consensus. Were we to treat exercise as we do diet, we would doubt, debate, refute, and ignore the health benefits of walking, running, hiking, biking and swimming- until we could say exactly which version of one could beat every specific expression of the others.

Such, then, is the bite of dietary dogma.

It goads us to relearn ad infinitum what we already know, and feign recurring surprise. It prods us to renounce what we know reliably for want of the “sanctioned” methods of inquiry and apprehension. It impels us toward the pure nonsense of a single answer to questions about diet and health for a blended population of those subject to obesity and type 2 diabetes, and those subject to marasmus, kwashiorkor, stunting, and overt starvation.

There is, however, one bit of heartening news in the mix. Dogma is brash, not stealthy; its bark rather reliably precedes its bite. When it does, cover your ears, and open your mind- and all should be well. There is even the possibility that you might hear yourself think.

-fin

This article was first published on LinkedIn.

Science without sense is train with no tracks; a train wreck is all but inevitable. Sense without science is tracks but no train; progress is slow, and much that matters is out of reach. Only from the pairing of science and sense is understanding of seemingly elusive truth likely to issue.

Black and White image of a derailed steam locamotive train

Facebook
Twitter
LinkedIn
Pinterest