A (very) simple nutrition study explainer
Trial types
The following hierarchy of evidence is widely cited and generally accepted. Meta-analyses and systemic reviews put together all the studies on a subject and critically review the results. Umbrella reviews are a review of reviews, either systematic reviews or meta-analyses. Meta-analysis also include a data summation to quantify the magnitude of effect (there’s an example further on). Randomised trials are held up as the gold-standard trial on which these analyses should be based.
The Hierarchy of evidence Source: Edith Cowan University (Ref)
Good in theory perhaps but conducting randomised controlled trials (RCT) in nutrition presents several challenges.
1. Blinding
The best type of RCT is a the randomised double-blind, placebo-controlled study. Here, subjects are randomly assigned to either an intervention or the placebo control. Both participants and researchers are unaware who is receiving the intervention and who is getting the placebo. That is, they are double-blinded. Double-blinding is a means to minimise pre-conceived ideas affecting the outcome. It is possible to blind a vitamin pill trial but less so a food intervention trial.
Meanwhile, randomised allocation attempts to minimise confounding factors, as it should prevent factors clustering in one group or the other as they tend to in real life (for instance, vitamin takers also tend to be health conscious, with health consciousness, confounding a “pure” observation of the health effects of vitamin taking).
An example of a double blind randomised nutrition RCT is the Alpha-Tocopherol, Beta-Carotene (ATBC) Cancer Prevention Study, where 29,133 middle-aged male smokers received either vitamin E or beta-carotene, or both, or placebo for a median of 6.1 years.
2. Time and sample size
The main difficulty is that many nutrition exposures only manifest their effects over decades, or at least years, rather than days or weeks and effects may be subtle. It’s not feasible to confine subjects to a metabolic ward to monitor adherence to a prescribed intervention for more than a few weeks, so longer-term trials are invariably conducted in free-living subjects. Trials that prescribe an intervention over years at a time in subjects going about their daily life, are often plagued by suboptimal adherence, which becomes more suboptimal as the trial progresses. An example might be assessing the effect of a vitamin supplement on a health effect. Researchers try to adjust for adherence by recruiting large numbers to start the trial and periodically monitor biomarkers, such as blood levels of the vitamin to assess adherence but it is possible that the result does not show statistical significance after the prescribed period because the trial is ultimately under-powered.
Here’s an example of a metabolic ward trial conducted over four weeks assessing the effect of a ketogenic diet on energy expenditure and body fat. The PREDIMED (Prevención con Dieta Mediterránea) is probably the most well-known example of a large long-term RCT in free-living subjects and evidence of waning adherence in the group assigned the low fat diet is evident, an effect which has been noted elsewhere.
3. Baseline exposure
Prescribing an intervention implicitly assumes that it’s a new exposure for a subject, which might be the case with a new pharmaceutical but will rarely be the case with a nutrition intervention. For instance, an omega-3 supplement trial assumes, subjects have a suboptimal background level of omega -3 (fish, supplements, possibly alpha-linoleic acid) intake. It’s possible to measure biomarkers to assess Omega-3 status but this reflects about three-months’ exposure and may not reflect life-long exposure to date, which is relevant in assessing health risks. The trial also assumes that placebo subjects will not consume significant omega-3 throughout the trial, which may be lengthy. This assumption may be false and can often muddy the waters.
4. Ethics
This previous example could also be used to explain the question of ethics. If researchers suspect omega-3 is health enhancing, is it ethical to deny subjects omega-threes? Occasionally trials may be stopped early because the signal for benefit or harm indicates subjects may be at risk. For example, the PREDIMED (Prevención con Dieta Mediterránea) trial was stopped after 4.8 years and all subjects advised to adhere to the Mediterranean Diet.
Mendelian Randomisation
Mendelian Randomisation can be a bridge between observational studies (with their confounding problems and questions over correlation versus causation) and RCTs and may finally move forward some questions in nutrition which have been the subject of heated debate due to the questionability of the evidence.
Mendelian randomisation (MR) is a research method that uses gene variants to test whether an association between a risk factor, for instance low-density lipoprotein cholesterol (LDL-C), and a health outcome, let’s say heart disease, is causal. Databases of genome-wide genetic data, such the UK Biobank, which includes half a million participants, have greatly facilitated these analyses.
MR can mitigate two problems that plague observational studies:
Confounding: For example, does high C-reactive protein (CRP) cause heart disease? Or does inflammation drive both CRP and heart disease? MR studies suggest the latter, which would make CRP a confounding factor. If CRP was causal, then gene variants that raise CRP would correlate with observed heart disease in the population.
Reverse causation: Does overweight drive high CRP or does CRP drive overweight? Here the former is true. High overweight genes correlate with both observed population overweight and observed raised population CRP, while higher CRP genes correlate with observed population CRP but not with population overweight.
And, yes, MR studies strongly support a causal role for LDL in heart disease.
P-values and confidence intervals
A p-value answers this question: If whatever we’re assessing actually doesn’t make any difference (the null hypothesis is true), what is the probability of seeing results at least as extreme as out outcomes?
A p-value of ≤0.05 is usually set as the benchmark for statistical significance. It says there is ≤5% chance the null hypothesis is true.
Confidence interval is a related concept.
The 95% confidence interval (CI) says you can be 95% confident the underlying true effect is included in a range. If your null hypothesis is outside your confidence interval, then you can be 95% confident in rejecting it. Here’s an example from a recent article. Stomach cancer is non significant as the 95% CI includes 1.0 (here, 1.0 is equivalent to no observed effect). Note that stomach cancer shows a non-significant trend for an effect of alcohol, whereas pancreatic cancer, in this analysis shows no effect at all.
Results of a meta-analysis of observational studies. The dose-response relationship for the risk of cancer at different sites per 10 g/day increase in alcohol consumption. RR = Relative risk; CI = Confidence interval. Source Rumgay et al. (2021) (Ref)
A theoretical example testing the hypothesis that average calcium intake is 750mg/day:
p-value: tells you whether your data support the hypothesis “average intake = 750 mg/day”.
Confidence interval: tells you your best estimate of the true average, for instance, “500 – 1300 mg/day”.
DALY, YLL, YLD? Please explain
These terms are very common are are used to standardise health risks in a population. They facilitate comparison between populations, countries, times and between risk factors. One application might be as a funding decision tool to see where money spent will have most impact.
Years lived with disability (YLD)
The years spent living with disease or injury due to a risk factor, that, in the absence of the risk factor would have been expected to be healthy. If a person might expect to live without disability until age 65 but becomes permanently disabled at 20 in a car accident (the risk factor), YLD would be 45 years.
Years of life lost (YLL)
Years lost due to early death, attributable to a risk factor. If, as a result of a car accident, a person dies at 65, where, in the absence of the car accident, life expectancy was 80 years, YLL would be 15 years.
Disability-adjusted life years (DALY)
The years of life lost through premature death or disability due to illness or injury.
DALY = YLL + YLD
