Introduction

We discuss in this review the role of health status as unmeasured confounder in studies regarding the effectiveness of influenza vaccination on health outcomes in older age group. We are reviewing the literature with major emphasis on article by Groenwold et al that discusses different perspectives of the current debate. It provides valuable insights into evaluating unmeasured confounders, functional health status in particular. It is valuable since it provides opportunity for examining the alternative explanations governing uncontrolled confounding. Using example of this article, we present this review with the objective of identifying and assessing unmeasured confounders. We present stepwise approach to identify and assess uncontrolled confounder in a given study.

How to identify confounders?

Separation of the effects of extraneous variables from the effects of a factor under study is one of the key prerequisites for validly estimating the magnitude of the study factor's effects. Further, the decision to label a variable, as a confounder in a data set must be made on the basis of subject-matter knowledge and clinical judgment and that there is no alternative to use of such judgment. For example, in the article by Groenwold et al 1 on the one hand they show that functional health status fulfills the criteria for being a confounder and on the other discount the need to adjust for it in analyzing the association between influenza vaccination and mortality in the elderly.

The assessment as to whether functional health status is a potential confounder and to further adjust for it should be guided by biological context and intuition without ignoring the methodological issues. First, prior information is most valuable criteria (independent of whether any statistical test proves it or not) in including functional health status as a confounder and to further assess the extent to which it would bias the association between influenza vaccination and mortality. In their systematic review Jefferson et al (2005) discuss that influenza vaccine has differential response with beneficial effect against complication among elderly in long-term care facilities and its usefulness in the community is modest. Further, they infer that difference in the results due to baseline imbalance in health status and other systematic differences in the two groups of participants cannot be discounted.

Second, in order to be a confounder, a given covariate must be an extraneous risk factor for the outcome in the sense its association with the outcome arises from a causal pathway other than one under study and be an independent risk factor for the outcome in the unexposed. . The design of the studies such as by Groenwold et al 1 does not allow the examination of these prerequisites for confounders. The covariate to be labeled as a confounder should also not be affected by either exposure or disease. It can further be argued that functional health status can be altered by influenza vaccine in the elderly (could be time-dependent), hence would satisfy to be an intermediate in the causal pathway to mortality, and thus, should not be treated as a confounder in the analysis.

The dilemma arises due to conflict between prior information stating that functional health status is a potential confounder and a possibility for it to be simultaneously a causal intermediate. Standard approaches for adjustment for confounding by such covariates are biased. Advanced analytic methods, such as G-estimation , would allow for adequate adjustment in such instances.

Even in the absence of prior information from other studies, one can still identify and quantify the bias due to unknown potential confounders. In the current situation this would require knowing the magnitude of association between functional health status and mortality within levels of immunization status, the immunization-specific prevalence of functional health status in the source population and the prevalence of immunized persons in the source population. The potential confounder is expected to completely account for the crude association given very high values of all the above parameters.

Statistical fallacies

Missing data about potential confounders in health databases or a result of differential non-response to items in questionnaires pose another challenge in estimating unbiased associations. In their article, Groenwold et al optimistically assume that the information from questionnaires is missing at random (MAR). Missing data can indeed be considered random conditional on showing the association between missingness and other characteristics that are available at the time of analysis. The imputations based on assumption of MAR will still lead to biased estimate of association. The bias resulted includes sparse data bias and selection bias. Let us assume that there is no residual confounding of functional health status in this study (which is pretty unlikely). Even then, multiple imputation under unjustified assumptions can lead to bias in any direction with a loss in ability to detect the actual effect of confounder on the observed association.

It may be noted that the statistical methods with respect to predictiveness and discrimination of models are generally ambiguous 1 Upholding predictiveness simply on the basis of a statistical test is usually far too insensitive to detect all the important confounders and resulting adjustment may produce highly confounded estimates of association. Further, C statistic and Tests of Fit have low power to detect all forms of model fitting . Large p-value leaves open the possibility for the regressors to be important in describing the true regression function but that the test failed to detect this condition. It may also mean that some terms that are not present in the reference model are important. Also, good fitting data does not guarantee the correctness of the model. Discriminatory value depends on the frequency of exposure and the outcome (Greenland S 2008).9 Even if a covariate is poorly associated (as a confounder) in the source population, it can produce high discriminatory value since this value is dependent on the frequency of exposure and outcome in source population (Greenland S 2008). It may not have anything to do with the exposure-confounder-outcome relationship and thus has no utility in scientific inference (Greenland S 2008).9

Groenwold et al 1 argue that stabilization of the effect estimate is a strong argument against residual, unmeasured confounding. We think that stability is a property of the results obtained from the observed data. The possibility of outliers and/or influential data points to be the reason for such artificial stability cannot be ruled out. However, even if there are no outliers, stability of effect estimate does not rule out residual confounding in this study as cancellation of confounding by covariates in opposite directions is still possible.

Functional health status is likely to be correlated to co-morbidities, prior healthcare and medication use within the same individual, as also suggested by Groenwold et al.1 However, this can result in collinearity such that the effect of functional status on the vaccination status might not be apparent because their combined effects could tend to cancel each other out. Highly correlated factors can not only affect the numerical accuracy of regression parameters but also make it difficult to interpret the direction and magnitude of their effects. Use of statistical techniques such Principal Components Analysis and Ridge Regression could assist in better interpreting these results.

Conclusions

The goal of this review is to stress the need for following clear steps for identification of potential confounders. Identification of potential confounders should be guided by biological context and intuition without ignoring the methodological issues. While “prior” information is important in such identification, other factors required for a variable to be a confounder include that the it has to be an extraneous risk factor, association with exposure among non-diseased and not being (a mediator or a collider) in causal pathway.

Articles such as by Groenwold et al 1,conclude that unmeasured confounders, such as functional health status, have no impact on the estimated effectiveness of influenza vaccination in reducing mortality risk. The attempt of this review is to infer that the readers would be benefited more if the authors spelt out the study limitations explicitly, including the rationale for their assumptions. We wish to submit that even with the best of study design and (perceived) lack of all bias, results from a single study cannot be used to draw final conclusions.