On Monday, Wired published an article about the Nutrition Science Initiative, “The collapse of a $40 million nutrition science crusade“.  This seems like a good time to review the trajectory of NuSI and what we can learn from it.  NuSI was founded in 2012, ostensibly on the idea that our current understanding of obesity is at best incomplete, and possibly incorrect, and we need larger and more rigorous experiments to tease out its true causes.  In practice, it was founded to investigate the carbohydrate-insulin hypothesis of obesity, the favored hypothesis of its co-founder Gary Taubes (its other co-founder is Peter Attia, MD).  It was clear in early materials released by NuSI that its staff believed strongly in this hypothesis and felt that these experiments would vindicate it and address the obesity crisis.  They predicted their efforts would reduce the prevalence of obesity by 50 percent, and the prevalence of diabetes by 75 percent, by 2025.

These grand ambitions, coupled with Taubes’s high-profile public advocacy stretching back to 2002, and Attia’s energy and vision, netted NuSI over 40 million dollars, almost all of it from the Arnold Foundation.  In 2012, I was contacted by Taubes and Attia, who explained what NuSI was and asked for my support.  As I explained in a post I wrote at the time, I ended up endorsing NuSI because I viewed it as an additional funding mechanism for testing questions related to obesity and nutrition that may otherwise be difficult to fund.  Equally important, I saw that they were selecting high-quality researchers to do the experiments rather than cherry-picking people who were on the home team.  I had a conversation with Kevin Hall, PhD, during which he reassured me that under the terms of the agreement, NuSI wouldn’t have the ability to tinker with study data or interpretation.  More funding for interesting science, and an agreement that prevents scientific tampering.  What could go wrong?

That said, I did express major reservations about NuSI at the time.  Their materials suggested that they were heavily biased, and, to quote myself, “It would be remiss of me not to question the wisdom of putting a major science funding mechanism into the hands of a journalist who is, shall we say, very attached to his ideas.”  Despite these reservations and my substantial uncertainty about whether NuSI would support good science or become a corrupting influence, I felt that it had more potential to do good than harm.

The first study NuSI funded was a meticulous eight-week pilot trial, led by Hall, to see if a very-low-carbohydrate (ketogenic) diet causes a meaningful increase in calorie expenditure vs. a calorie-matched standard American diet with more than tenfold the sugar.  Volunteers were kept in a research facility where every morsel they ate was controlled.  The study design was agreed upon in advance by both the researchers and NuSI, and the diets were designed by low-carbohydrate diet researcher Jeff Volek, PhD, RD.  According to Taubes, calorie intake is virtually irrelevant and insulin is everything, so his hypothesis predicts large and obvious differences in calorie expenditure and fat loss between diet periods, despite equal calories.  The results showed little effect of the ketogenic diet on calorie expenditure, and an initial slowing of fat loss, despite a large and persistent decline in insulin secretion.

NuSI reacted poorly to this news.  They suddenly became very critical of the study’s design, and according to a timeline shared by Hall, “requested access to the data and began providing extensive comments on the data analyses and interpretation”.  NuSI also requested that the researchers withdraw an abstract on the study they had submitted for the American Diabetes Association meeting, which they did.  I find this particularly galling. [Update 8/22/18: the reason for this is more complex than I was initially aware of.  See this post for an explanation.]

The study did have substantial limitations, most notably its lack of a true control group*.  After seeing the study’s results, Taubes began arguing that these limitations make the study uninformative, reversing his initial support for its design.  The truth is that, despite the study’s limitations, there is no way to reconcile its findings with Taubes’s version of the carbohydrate-insulin hypothesis.  If carbohydrate and insulin do what Taubes thinks they do, the extreme difference in carbohydrate and sugar intake between the two experimental diets would have made this clearly visible.  NuSI seemed to understand this before the results came in.

The interference from NuSI became so intolerable that in April of 2015, Hall and the other researchers wrote an email to NuSI asking to “reestablish the academic freedom” of the research team, as per their initial agreement.  Rather than backing off, NuSI doubled down.  In June of 2015 it “confirmed that it wanted to be more closely involved in the design, analysis, and interpretation” of the next study.  This is documented in Hall’s timeline.

From there, NuSI entered free fall.  Attia resigned in December of 2015.  After a spat between Hall, Taubes, and Mark Friedman, MD in front of John Arnold in January of 2016, which presumably broke the camel’s back, Hall resigned from his NuSI-supported position.  The Arnold Foundation stopped sending checks, and NuSI withered.  NuSI still exists today, but most of its staff are gone and it’s limping along seeking funding. [Update 8/22/18: the reason for the collapse of NuSI is more complex than I was initially aware of.  I spoke with Peter Attia and painted a more complete and accurate picture in this post.]

In retrospect, it’s not surprising that NuSI collapsed under the weight of its own ideology.  The nature of ideology is that it’s inflexible when it encounters facts that contradict it**.  It cannot bend, it can only break.

One thing that I think hasn’t gotten enough attention is the fact that NuSI staff, particularly Attia, paid themselves enormous salaries while they were at NuSI***.  According to public records, Attia received an average of $425,000 per year for four years, including $727,754 in 2015, for a grand total of $1.7 million.  Taubes received an average of $117,000 per year for five years, netting $586 thousand.  Friedman received over $200,000 per year in 2014 and 2015, netting $495 thousand.  [Update 8/22/18: After speaking with Peter Attia, I now believe there was nothing fishy about their salaries.  They were set by an independent board, not by NuSI execs themselves.  See this post for more detail.]

NuSI is a 501(c)(3) nonprofit corporation, meaning that its funding is explicitly intended to benefit society, not individuals.  People have to make a living of course, but it’s hard to understand the rationale for a $728k salary at a nonprofit.  Although Taubes’s salary appears more reasonable than Attia’s, he was writing and promoting The Case Against Sugar and doing other non-NuSI work during this time, so it appears to be for part-time work.  I’m not going to come to any grand conclusions about this before hearing their side of the story, but they certainly have some explaining to do.

Here are my takeaways from the NuSI experiment, with the benefit of hindsight:

• Obesity is a difficult problem to solve.  Many grand ambitions have foundered on this fact.
• As a result of NuSI’s funding, useful research was published, and more potentially useful research is forthcoming.  So I don’t think it’s correct to view NuSI as a failure, despite the fact that its funding and staff have mostly evaporated.
• Certain important norms in science only work as long as everyone is behaving themselves, and this makes science fragile.
• Ideology and science don’t mix.  The risk in giving ideologues power is not simply that they may fail, but that they may fundamentally corrupt science.

 

* One could argue that the subjects were their own controls, but ideally for this to work it should be done in counterbalanced crossover fashion with a wash-out period between diets.

** Philip Tetlock, PhD describes ideologues beautifully in his book Superforecasting.  He refers to them as “hedgehogs” and pragmatists as “foxes”.  Hedgehogs are persuasive because in their eyes, the world is simple and they are utterly convinced they know how it works.  But they are abysmally bad at making predictions because the world isn’t as simple as they think.  Most television pundits are hedgehogs and their predictions are about as good as flipping a coin.  Foxes, on the other hand, understand that the world is complex and try to factor this complexity into their worldview.  They learn from their mistakes.  As a result, they are better at making predictions.  But they get less attention because they’re less confident and their view of the world is harder to explain.

*** I initially learned about this in a post by Evelyn Kocur.  I don’t endorse the aggressive tone of the post but she certainly did her homework.  I have independently confirmed the salary figures in her post.

I’ve always been struck by the imprecision of how the terms “low-glycemic-index diet” and “low-glycemic-load diet” are used.  In theory, these are diets that reduce post-meal blood glucose and their benefits derive from this property.  In practice, they often differ from typical diets in multiple ways.  If a “low-glycemic diet” is also higher in fiber and protein and/or lower in palatability, calorie density, and carbohydrate, can we really attribute its beneficial effects to its impact on blood glucose per se?  To gain a better understanding of the depth and pervasiveness of this problem, I did a scientific literature search for randomized controlled trials of low-glycemic-index/load diets, selected the first ten I found, examined the details of the assigned diets, and evaluated the findings.

Methods

I performed a search in Google Scholar using the search terms “randomized trial low glycemic”.  I scanned down the list until I had identified the first ten studies that met the following criteria:

• Randomized controlled trial
• Compares a low-glycemic-index or low-glycemic-load diet to a control diet
• Title describes the low-glycemic diet only according to its glycemic index/load, not other diet characteristics (e.g., high protein, legumes)
• Any length (single meal to long-term diet trial)

I extracted the following information from each study:

• First author
• Year published
• Title
• Summary of subject characteristics
• Length of intervention
• Diet summary
• Detailed diet description
• Was the between-diet comparison well controlled for glycemic index/load?  Yes = differences in glycemic index/load were achieved without substantial differences in the types of foods consumed.  Moderate = the types of foods differed substantially between groups, but major nutritional characteristics like protein and fiber intake were well controlled.  No = major nutritional characteristics like protein and fiber intake differed between groups.
• Outcome of the intervention

My primary outcome was how well each diet was controlled for dietary variables other than glycemic index/load.

Results

I compiled a table of the findings here.  Trials were published between 2004 and 2011, and were all highly cited (105 to 413 citations according to Google Scholar).  They lasted between 4 weeks and 18 months, and 6 of 10 involved subjects with diabetes.

Among the ten trials, five were clearly not controlled for glycemic index/load (“no”), four were moderately well controlled (“moderately”), none were well controlled (“yes”), and one didn’t provide enough information to judge.

Overall, the results of the trials suggest that low-glycemic diets can be helpful for blood glucose control in diabetes, but their effects are modest in people without diabetes.  They are not very effective for weight or fat loss (relative to control diets, which varied) and they seem to have little impact on birth outcomes in the context of gestational diabetes.

Discussion

In this limited but unbiased sample of highly cited papers on low-glycemic diets, I found that none isolated glycemic index/load as a variable, 40 percent were moderately well controlled for glycemic index/load, and 50 percent were poorly controlled for glycemic index/load.  Another way of stating this is that all of the diet comparisons differed in ways other than glycemic index/load, and five of them differed in major ways that are likely to be confounding.

This is only a problem if we attribute the outcomes of these studies to glycemic index/load per se.  If we all understand that “low-glycemic diet” often means “beans, nuts, fresh fruit, and slow-digesting whole grains that may improve health via multiple mechanisms relative to typical diets”, there’s no problem.  This is the interpretation I believe is most accurate, but it’s not what I see in most discussions of these studies.

The findings of this brief literature survey suggest that we should be cautious about attributing the effects of low-glycemic diets to their impact on glycemia per se.

 

***April Fools post***

Let’s face it: most juicers are terrible.  They’re complicated, hard to clean, expensive, and they break down.  Worst of all, from a nutritional standpoint they waste important nutrients, especially fiber.  What if there were a juicer that solved all these problems at once?  I’ve finally found it: the IllaMAX 3000.

The IllaMAX 3000 juices apples, celery, berries, kale, and all my favorite nutrient-packed foods with ease.  It can even make smoothies!  You get the delicious juice, plus all the nutritional benefits of the fiber.  And you don’t end up having to throw away the leftover pulp because all it’s in the juice.  How can this be possible?

Two other remarkable facts about the IllaMAX 3000.  First, unlike other juicers, it’s self-cleaning, so you’ll never have to clean out the pulp again.  Second, it’s so incredibly durable it can last up to 122 years.

This might seem too good to be true, but it’s made possible by some very advanced technology:

• Rock-hard hydroxylapatite grinding stones are built for a lifetime of use
• Two powerful mechanical levers apply crushing and grinding force to the stones
• A reliable peristaltic pump collects the juice

You won’t believe the price.  Not $199.99.  Not $99.99.  Not even $49.99.  But absolutely FREE!  In fact, you may even already have one installed!

April Fools!

On the Amazon page for my book The Hungry Brain, the top review is positive overall but regrets that I didn’t cover the role of the gut microbiota in obesity.  This reflects a common belief that differences in the microbial composition of the human gut are an important determinant of body fatness.  In fact, the omission in my book was deliberate, because despite the popularity of this idea, I’m still not convinced it’s correct.  In this post, I’ll review three recent findings that have added to my skepticism, including a human fecal microbiota transplant study that doesn’t mean what most people think it means.  I’ll note up front that I don’t think this evidence proves that the gut microbiota isn’t involved in determining body fatness– it just proves that the hype is outpacing the science.

Finding #1

The first finding is actually a reanalysis of a seminal result in the microbiota-obesity field.  The initial paper was published in the journal Nature in 2006, and to large extent, it built the field.  The key experiment shows that a fecal microbiota transplant from an obese mouse to a lean (microbiota-free) mouse can make the latter gain fat.  This was an important milestone because it demonstrated for the first time, apparently unequivocally, that the microbiota was actually causing part of the fattening effect– not just along for the ride.  The study has been cited 5,931 times.

I’ve always been skeptical of this paper, for different reasons, but recently a microbiome researcher named Matthew Dalby took a closer look at the data underlying the claim that body fatness can be transferred via the microbiota and published the results on his blog.  I won’t get into the details of his analysis– you can read them on his blog if you’re interested– but suffice it to say the finding doesn’t survive critical inquiry.  The effect size is trivial and it’s likely explained by a statistical artifact called regression to the mean [Update 3/27/18: Jeffrey Walker has done a detailed statistical analysis of this experiment and shown that it is indeed likely explained by regression to the mean].  This type of anomaly is one of the hazards of using small groups of animals in your experiments (9 – 10 per group in this case), and it was exacerbated by the way the researchers chose to report the data.

This undermines a seminal finding of the microbiota-obesity field and increases my skepticism about the hypothesis as a whole.

Finding #2

The second finding is also from Matthew Dalby, but this time in the form of a study.  Published late last year in Cell Reports, the study compares body fatness, metabolic health, and gut microbiome of mice fed three different diets:

1. Unrefined low-fat diet.  This is the typical maintenance diet for rodent colonies and it’s primarily composed of unrefined grains and soybeans.
2. Refined low-fat diet.  This diet is composed mostly of refined carbohydrate, largely corn starch and maltodextrin, but also a bit of sucrose.  It contains casein for protein and a few other ingredients.
3. Refined high-fat diet.  This is the typical style of diet that’s used to produce obesity in rodents, including in my own experiments.  It’s mostly lard by calories, but also contains corn starch, maltodextrin, and a bit of sucrose.  It contains casein for protein and a few other ingredients.

There are many interesting findings in this paper, but for our purposes there is one key result.  The mice fed the unrefined low-fat diet and the refined low-fat diet remained lean and metabolically healthy (good glucose tolerance), while the mice fed the refined high-fat diet quickly became obese and metabolically unhealthy (poor glucose tolerance)…

 

 

…Yet when they examined the gut microbiota, it was similar in the mice eating the refined low-fat and refined high-fat diets (while the microbiota on both refined diets were quite different from the unrefined diet).  Therefore, differences in the microbial composition of the gut flora couldn’t explain differences in body fatness or metabolic health.

Finding #3, or how not to analyze data and report study results

The ultimate proof that gut microbiota impacts human body fatness would be a study demonstrating that a fecal microbiota transplant from lean people to people with obesity can cause weight loss, or vice versa.  The first study to examine the weight impact of a fecal microbiota transplant from lean people to people with obesity was recently published— let’s have a look.

The first place we’ll go is to the study’s preregistration site.  This is the place where the authors laid out their research plan prior to conducting the study, including how the data would be analyzed an interpreted.  The reasons authors preregister research plans, aside from the fact that it’s often required by quality journals, is that it shows that they didn’t tweak their analysis and reporting after seeing the data, in order to engineer a preferred outcome.  This can lead to results that are convenient for the researchers but misleading for the scientific community.  Preregistration is a critically important tool for combating the “replication crisis” that is currently sweeping many fields of research as we realize their findings aren’t as reliable as we thought they were.

One of the most important things a researcher should preregister in any randomized trial is the study’s “primary outcome”.  This is the outcome that will matter the most for study design, data analysis, and interpretation.  It is the outcome that researchers commit in advance to focusing on in the resulting publication, no matter what the results are.  Researchers can also register “secondary outcomes”, which are less important.  In this case, the preregistered primary outcome was “changes in weight in relation to fecal flora composition and short chain fatty acid metabolism in fecal samples after 3, 6, 12 and 18 weeks”.  This isn’t very specific but it’s clear that the primary outcome is about changes in body weight.  Thus, according to the researchers themselves, the study was designed and committed in advance to be primarily about body weight.  Insulin resistance is listed as a secondary outcome.

Onward to the paper.  The title should immediately send up a red flag: “Improvement of Insulin Sensitivity after Lean Donor Feces in Metabolic Syndrome Is Driven by Baseline Intestinal Microbiota Composition”.  The title focuses on a secondary outcome but doesn’t mention the primary outcome of body weight, and the abstract does the same.  In fact, the paper barely mentions that the intervention had no impact on body weight at the two reported time points of 6 and 18 weeks, and all the figures relating to it are in the supplementary materials.  Let’s not be misled by this sleight of hand: the study was primarily about the impact of fecal microbiota transplant on body weight, and it found no effect.

The paper does claim that lean microbiota improved insulin sensitivity at 6 weeks, but this result is unconvincing due to poor statistical methods.  I’ll relegate my discussion of this to a footnote so I don’t bore you with technical details.*  I don’t know what the reviewers were smoking while reading this paper but it must have been strong stuff.

Conclusion

We still don’t have compelling evidence that differences in the composition of the gut microbiota significantly impact body fatness in humans, and the best human evidence we have suggests that it may not be important.  New evidence also suggests that it may not be as important in rodents as we thought either.  That said, the case isn’t closed yet.  The human evidence we have is short-term, and given the staggering complexity of the microbiota and how it interacts with diet and lifestyle, there is still room for it to be important.  I look forward to further research on it, but in the meantime, let’s cut back on the hype.

Thanks to Matthew Dalby for his thoughts on this post.

 

* They used within-group comparisons to show that the group receiving the lean-type microbiota showed increased insulin-mediated glucose uptake over the 6-week period, but the control group didn’t.  They interpreted this as showing that the fecal transplant improved insulin sensitivity.  The problem with this is that key statistical tests in randomized controlled trials have to directly compare between groups, not within groups.  Otherwise, why bother having a control group?

I’ve been eagerly awaiting this study.  It was a 12-month randomized controlled trial comparing non-calorie-restricted low-fat and low-carbohydrate diets.  Both diets placed a major emphasis on diet quality.  I’m not going to review the findings in detail because Examine.com has already beat me to it.  But here are some highlights:

• Participants achieved large differences in fat and carbohydrate intake for the duration of the 12-month trial, although the difference narrowed somewhat over time (as it does in every diet trial).
• The first primary (most important) outcome was about insulin.  Genes influencing meal-related insulin secretion were unrelated to a person’s response to the low-fat or low-carbohydrate diets.  [Correction 2/21/18: These genes aren’t directly related to insulin; they’re more closely related to fat metabolism and previous preliminary evidence suggested they might impact the response to low-fat/carbohydrate diets]
• The second primary outcome was also about insulin.  A person’s baseline insulin response to carbohydrate was unrelated to his/her weight loss response to the low-fat or low-carbohydrate diets.
• Both diets caused similar weight and fat loss at 12 months (-5.3 kg [-12 lb] for LF vs. -6.0 kg [-13 lb] for LC).  These are better 12-month results than most diet trials.

Here are a few reasons why this study is a big deal:

• It was funded by the Nutrition Science Initiative, a nonprofit involving Gary Taubes that has been funding additional research to test the carbohydrate-insulin hypothesis of obesity.  This is the second of three studies NuSI has funded.  The first study showed that a very-low-carbohydrate ketogenic diet doesn’t accelerate fat loss or meaningfully increase metabolic rate relative to a high-carbohydrate, high-sugar diet of equal calories and protein, despite large differences in insulin levels (implying that ketogenic diets promote weight loss by causing people to spontaneously eat fewer calories).  So far, these studies are doing a remarkably effective job of further refuting the carbohydrate-insulin hypothesis.
• It was large and well-designed.  In included 609 people and the research plan was registered in advance, including which outcomes would be the most important (primary).  This guarantees that the researchers couldn’t tweak the methods after the fact or use selective outcome reporting to obtain a preferred outcome.
• It didn’t involve deliberate calorie restriction in either group.  One of the (valid) complaints of the low-carbohydrate community is that in randomized trials, non-calorie-restricted low-carbohydrate diets are often compared with calorie-restricted low-fat diets.  Some have speculated that if low-fat dieters were allowed to eat until full rather than controlling portion size, they wouldn’t lose weight, and they might even gain.  This is not the case.
• It focused on diet quality in both groups.  All participants were told to focus on unrefined foods and cook at home whenever possible.  This is in contrast to the wimpy low-fat control groups in many randomized trials of low-carbohydrate diets, which I call the “Snackwell diet”.

One typical (somewhat valid) critique about randomized trials like this is that the low-fat diet contained too much fat, or the low-carbohydrate diet contained too much carbohydrate.  In this study, both arms were started on very-low-fat (20 g/d) or very-low-carbohydrate (20g/d) diets, and told that they should keep fat/carbohydrate to the lowest level they can sustainably maintain.  While it is true that fat and carbohydrate intake rose throughout the 12-month period, this isn’t an indictment of the study: It’s a fact about human nature.  This trial showed that even with extensive support, the average person is unable to stick with a very-low-fat or very-low-carbohydrate diet.  It is true that they probably would have seen better results on both diets if they had been able to stick with a greater degree of restriction, but the participants demonstrated with their forks that it wasn’t sustainable for them.

Sugar intake declined substantially in both groups, although more in the low-carbohydrate group.  I predict that those who are constitutionally unwilling to give up the insulin hypothesis will focus on this as a way to discount the results.  Please allow me to inoculate you against this weak argument with the following facts:

• Sugar has little impact on insulin levels relative to other types of carbohydrate unless it’s consumed in unusually large amounts.
• High sugar intake doesn’t seem to impair weight loss, both in settings where calories are strictly controlled, and in settings where they aren’t.

Sugar probably does contribute to obesity, but the evidence suggests it’s very unlikely to be the primary driver of changes in body fatness.  I have no illusions that this study will change the minds of the diehards, but I do expect it to impact everyone else.

 

 

My book, The Hungry Brain, covers the science of eating behavior and body fatness and offers some tools for putting that evidence into practice.  Yet The Hungry Brain isn’t really a weight loss book because it’s more focused on understanding body fatness than on doing something about it.  The practical arm of my work is the Ideal Weight Program, a web-based weight management system that I co-designed with Dan Pardi, a researcher and the CEO of HumanOS.  I haven’t talked about it much over the last few years because we’ve been working on a major upgrade.  As is often the case, it took longer than anticipated.

That work is now complete and I’m happy to announce a new, much-improved version of the Ideal Weight Program.  As before, the program focuses on creating and maintaining awareness of key weight-related behaviors through education, tracking, and feedback.  Yet the new version delivers many upgrades:

• Free weight insight score.  This questionnaire helps you gain insight into the weight-related behaviors that are holding you back from your goals, and which changes may benefit you the most.  It’s available to anyone, free of charge.  Results are instant and you don’t have to provide your email address to receive them.
• The Ideal Weight Program is now course-based.  That means it’s divided into courses and lessons with corresponding mini-quizzes.  This low-stakes grading system increases engagement and the retention of information.
• Aesthetic upgrades.  We worked with professional designers to create clear, beautiful content.  The program looks much more appealing than the last version.
• A greater focus on weight maintenance.  I felt that the last version of the Ideal Weight Program didn’t offer enough support for weight maintenance– an important goal for many people.  In the new version, we’ve fleshed out our Lean Maintenance Diet, which is a Mediterranean diet modified for weight control.  Most of the courses in the current version apply to weight loss and maintenance equally.  The Ideal Weight Program is now a good value even if weight maintenance is your only goal.
• The Ideal Weight Program dovetails seamlessly with The Hungry Brain, particularly chapter 11.

The Ideal Weight Program is now part of the HumanOS platform, which is a greatly improved version of the former Dan’s Plan platform.  We’ve also switched over to a subscription model in which subscribers pay a monthly fee for access to all courses on the HumanOS platform, including the Ideal Weight Program (people who previously purchased the Ideal Weight Program will maintain lifetime access to the program with no monthly fee).

HumanOS offers many great courses and the number is growing–take a look!