As many of you know, a recent paper published in Cell Journal ( http://www.cell.com/cell-metabolism/abstract/S1550-4131%2814%2900062-X ) caused quite a stir in the fitness community. The paper’s conclusion was very provocative and led many in the media to make egregious statements such as “High protein diets as bad for health as smoking” as seen in this article: http://www.telegraph.co.uk/science/science-news/10676877/High-protein-diet-as-bad-for-health-as-smoking.html
Unfortunately these associations are based on faulty control groups, improper statistical analysis, misinterpretation, and misrepresentation of the data. My advisor, Dr. Donald Layman and a host of other experts on protein metabolism, critiqued this paper in a letter to Cell Journal, which Cell REFUSED to publish and instead recommended that the researchers publish their comments online. This is wrong on so many levels. In my opinion and the opinion of many others, this paper was NOT adequately critiqued and now Cell is avoiding allowing proper post publishing criticism of the paper in the journal. In my opinion this is not appropriate. Thus, I am publishing their comments on my website below. Enjoy.
The Contrived Association of Dietary Protein with Mortality
Donald K Layman, Arne Astrup, Peter M Clifton, Heather J Leidy, Douglas Paddon-Jones, Stuart M Phillips
On March 18, we submitted the following Letter to the Editor (LTE) concerning the article by Levine et al (Cell Metab 2014;19:407). This article contains numerous errors in design and analyses; and the authors present biased conclusions both in the article and in University press releases on the article which equated a higher protein diet to smoking. In our opinion the peer-review system has failed to adequately evaluate this paper. When this happens, the scientific community has the responsibility to provide additional oversight with scholarly evaluation and debate. However, the Editors of Cell Metabolism declined to publish our LTE and recommended that we post our comments on the Journal website. We view the decision to confine our views on this paper to online discussion forums as a severe limitation to academic discussion and debate that completely minimizes alternative evaluations. We view the position taken by the Editors as inconsistent with goals for maintaining scientific integrity. With this view in mind our unpublished LTE is provided below.
Dear Editor:
We applaud efforts to improve human health by asking insightful questions that explore existing nutrition paradigms. Unfortunately, the paper by Levine et al. is a flawed attempt to link health risks of a single nutrient, protein, to chronic disease states of cancer, CVD and diabetes. The study design and analyses are inappropriate; key contradictory data are neglected; and conclusions are not justified by the data. As scientists with decades of experience studying the impact of protein on health, we are concerned that translation of these flawed data and exaggerated conclusions to the public could have serious negative health consequences for adults seeking to maintain muscle health and avoid sarcopenia.
The optimum dietary intake of protein for adults remains a topic of scientific debate; however, research indicates that balanced diets with protein intakes moderately above the RDA value of 0.8 g/kg/d are beneficial for weight management, sarcopenia, diabetes and physical activity. While data overwhelmingly demonstrate short-term benefits of moderate protein intake on metabolic status and body composition, the long-term impact of protein on disease risk or mortality is more difficult to assess and requires expert interpretation of large data sets such as the National Health and Nutrition Examination Survey.
In their study, Levine et al. indicate (Figure 1, Table S1, and Discussion) that “…the level of protein is … not associated with differences in all-cause, cancer, or CVD mortality.”In fact the data demonstrate that cancer mortality was actually ~10% higher in the low protein group compared with the higher protein group (ie. 9.8% versus 9.0% deaths). We would argue that these obvious findings are the most important.
Subsequent subdivision and reanalysis of the data raise serious questions about the validity of the authors’ approach and conclusions. First, the NHANES Linked Mortality Files contain information for almost 12,000 adults, however, without justification, the investigators eliminated almost one-half of the data and only reported results for 6,381 over the age of 50 yr. Second the investigators created ill-defined protein groups of low (LP: <10% of kcal), medium (MP: 10 – 19.9%) and high (HP: 20%+). As defined by the Institute of Medicine (IOM) the Acceptable Macronutrient Distribution Range for protein is 10% to 35% of daily energy intake; thus, the LP group (<10% of kcal) consuming ~41 g/d (Table S1) should be designated as protein inadequate. Applying the authors’ unusual protein categories resulted in only 437 individuals remaining in the LP group. A third major problem is use of only a single 24-h recall to derive dietary data to represent food intake over the 18-yr period of life. The limitations of this approach are hard to overstate. NHANES surveys contain additional dietary recall data allowing for calculation of more meaningful “estimated usual” food intakes as utilized by other investigators. A fourth major problem is failure to report body weight or BMI for the groups. Energy balance and body fat are major risk factors for mortality from diabetes, cancer and CVD. The footnote for Table S1 contains the definition for the BMI abbreviation but the table omits the data.
The investigators also looked at the diabetes mortality data. They report a trend for increased risk of diabetes mortality for adults with higher protein intakes (Fig. 1). However, these findings are derived from very limited data. For diabetes mortality, they report 1.0% deaths (Table S1) or a total of 68 deaths in the entire population with only a single death in the LP group. Further, 47 of the individuals who died from diabetes had diabetes at baseline, before the first dietary measurements. The very low occurrence frequency increases the probability of statistical errors resulting in differences that are not biologically true; nonetheless, the investigators used these limited data to conclude “high protein was associated with … a 5-fold increase in mortality.” Further, the authors used Hazard Ratio (HR) analysis and concluded that the HP group had a 73-fold increased risk of dying from diabetes. The HR and confidence interval (CI) were reported as 73.52 (4.47 – 1,209.70). To our knowledge, that is the highest HR ever reported for any dietary component and certainly for one within dietary guidelines of the IOM. The CI with a 400-fold range and an upper value of 1,209.70 with 6-significant figures of accuracy is not credible.
Hazard Ratio analysis is a standard method for clinical studies with equal treatment groups and survival as a primary outcome, but have important a priori criteria for their use: 1) equal size groups, 2) no evidence of selection or group bias, and 3) linear outcomes over time. The present study fails to meet all three criteria. There are recent high quality papers using the same NHANES data focused on sugar and sodium/potassium using acceptable methods. The authors should justify not following established procedures.
Our overall assessment of this paper is that the conclusions and analyses are biased, and flawed. While there is growing consensus that a moderate protein intake between 1.0 and 1.5 g/kg/d may confer health benefits beyond those afforded by the current RDA for protein, we also recognize there are gaps in the current knowledge base and encourage discussion of important contradictory evidence/data. Future research must be well designed, rigorously reviewed, and credibility communicated. Unfortunately, the article by Levine et al. presents conclusions not supported by their own analyses or the greater literature
