Update: The recent Bell and Jones’ paper summarizes the debate which occurred to be still on: https://link.springer.com/article/10.1007%2Fs11135-017-0488-5
Discussion between Reither et al. (Yang and Land among authors) and Bell and Jones
This is a rare case of a long discussion between authors: there were four papers responding to each other by Reither et al. and Bell and Jones. The discussion is very recent, so maybe it hasn’t ended yet.
(Actually, discussion appeared even earlier. It began with seminal Yang and Land’s 2006 paper, where they suggested to model period and cohort effects as a cross-classified structure with age at the individual level. Several responses from Bell and Jones followed, for example their 2014 paper was titled “The impossibility of separating age, period and cohort effects”, in which they clarified why the old task of identifying Age = Period + Cohort equation cannot be solved neither theoretically nor practically.)
“Conventional solutions to the identification problem [in APC analysis] include:
1) setting equality constraints on at least one set of parameter estimates for age, period or cohort; and
2) linear nonparametric transformations, such as polynomial terms of age effects in repeated cross-sectional APC analysis.” (Reither et al., 2009)
Yang and Land (2006) proposed hierarchical age-period-cohort (HAPC) model. HAPC model is a cross-classified two-level model, where periods and cohorts are grouping variables for the second level and age is predictor at the first level. Random effects of period and cohort are claimed to be robust, so HAPC model “completely avoids identification problem”. In order to create a grouping variable for cohort, a birth year usually needs to be grouped. This groupings “also function as equality constraints, since each birth cohort within the interval is constrained to have the same effect.” It is argued that the groupung resolves collinearity issue, by imposing constraint. Bell & Jones (2014) add that “…numerous supposed solutions to the identification problem have been proposed, each of which imposes some kind of constraint on the model (Mason et al., 1973; Sasaki & Suzuki, 1987; Tu et al., 2011; Yang et al., 2008).The problem arises when these constraints are not clearly stated”.
After the inroduction of Yang and Land’s HAPC model, many papers applied this method without discussing possible false solutions.
Reither et al., 2009 used the model to decompose a growth of obesity rates in U.S. They grouped birth cohorts into 5-year intervals, used age and age squared at the first level, and estimated all the random intercepts (not just variance but estimates for each cohort and each period). The data are cross-sectional samples of millions of respondents. The model was estimated in several demographic subsamples.
They claimed that “This methodological approach enabled us to demonstrate that secular [i.e. period] changes are principally responsible for the obesity epidemic, while simultaneously showing that the odds of obesity have increased for recent birth cohorts.”
Bell & Jones (2014) doubted that. They claimed that the arbitrary selection of cohort intervals of 5-years is one of these constraints, and it should be theoretically underpinned (and Reither et al. hadn’t done it). The interval is actually an assumption that cohort effect is constant during this period.
That’s why there is a need for a solid theory, and since other papers suggested that period is not prevalent over cohort change in predicting the growth of obesity the results of Reither et al. might be not correct.
Bell & Jones (2014) simulated a data similar to what Reither et al. have found:
Logit(p=1) = -2 + 0.1age – 0.001age_sq + 0.04cohort + u(cohort) + u(period)
(it differed from Reither et al.’s model by excluding period effect, and substituting non-linear cohort effect with a linear one) and repeated HAPC analysis defining cohorts in three different ways:
– no grouping in the model, nor in data generation process (DGP or true values),
– 5-years grouping in the model, no grouping in DGP,
– 7-years grouping in the model, 5-years grouping in DGP.
None of the models reproduced the true effects. Hence, HAPC model didn’t show validity.
Reither et al. 2015 argued that Bell and Jones (2014) were not right, because they 1) missed period effect in their simulations, and 2) simulated linear effect of cohort instead of non-linear. The identification is not that bad if cohort is not linearly related to period and age, and it is usually not. Bell and Jones simulated linear relations that are unrealistic. So Reither et al. made their own simulations to show that the HAPC model finds true parameters when there is no linear trend in cohorts and periods and both can be regarded to be categorical variables.
Bell & Jones (2015) agreed that linear trends are rare in real life. However, “in many cases” it doesn’t work. Such cases are linear trends/dependence between A and C or continuous trends between P and C. “A linear trend exists when, if an algebraically linear expression is removed from the data at hand, this would have the effect of flattening that trend”. Further simulations confirmed that HAPC failed to predict the true APC parameters even when P and C are truly categorical.
Additionally, Reither et al. 2015 suggested estimating A, AP, AC and APC models and comparing their fit and then choosing the one with the best fit (BIC for example), as to be sure which effects are actually present in the data. Bell and Jones (2015) agreed, yet pointed out that fit statistics only suggest that there is significant non-linear variation present in each of A, P, C.
Reither et al. 2015 argued strongly in favor of random effects against fixed ones. Bell & Jones (2015) agreed – random effects should always be preferred over fixed for cohort and period. However, it doesn’t guarantee that the results are flawless. Finally, Reither et al., 2014 suggested that HAPC model “assumes zero correlation between the individual-level regressors and the random period and cohort effects. <…> this assumption can be assessed by application of a Hausman-type chi-square test. If the zero correlation assumption is rejected, then the model specification can be modified by treating either the time period or cohort effects (or both) as fixed.”
Fruits of the discussion
Summing up the points on which the authors agree, there are several suggestions in using HAPC.
- HAPC is not a universal cure for identification problem in APC analysis, however, it may work sometimes and it should be preferred over traditional methods like Mason’s one (Mason et al., 1973).
- HAPC needs a strong theory to be sure that:
a) “there are no trends in the periods or cohorts <…> there are not equal and opposite linear period and cohort trends (which, would cancel each other out, with the model estimating a spurious age effect rather than the true period and cohort trends) <…> “Where trends do exist, one option would be to make a decision based on theory as to which of periods or cohorts are most likely to have generated the data, and include that term in the HAPC model as a linear fixed effect (Bell & Jones, 2013a). This decision cannot, however, be made on the basis of the data” (Bell and Jones, 2014);
b) a grouping of birth years to cohorts is crucial and should not be arbitrary, they should rather be based on history. This is important theoretically: indeed, cohorts are not related to age and period exactly, cohorts by definition are some groups of population sharing the same socialization experience, historically (not by year itself) determined. I can add that age and period are also disconnected from year-by-year difference, they are grouped as well. In case of age/life course, groupings are related to psychological and physiological processes of ageing and growing up. In case of period changes, grouping is related to historical events. The significant events as well as growing up don’t happen every year. This is particularly problematic in cross-national analysis, since cohorts have very different experiences in different countries and lines of events differ a great deal.
3. Technical tricks should be applied. Though they do not guarantee anything, they may work in some situations:
- Random effects should be preferred over fixed effects model.
- Try estimating A, AP, AC and APC models and compare their fit and then choose the one with the best fit;
- Check with chi-square test whether there is a zero correlation between the individual-level regressors and the random period and cohort effects
- In case of a strong theory regarding age effect, Bayesian method could be used with informative priors about age.
Bell, A., & Jones, K. (2015). Should age-period-cohort analysts accept innovation without scrutiny? A response to Reither, Masters, Yang, Powers, Zheng and Land. Social Science & Medicine, 128, 331-333.
Bell, A., & Jones, K. (2014). Don’t birth cohorts matter? A commentary and simulation exercise on Reither, Hauser, and Yang’s (2009) age–period–cohort study of obesity. Social Science & Medicine, 101, 176-180.
Bell, A., & Jones, K. (2013). The impossibility of separating age, period and cohort effects. Social science & medicine, 93, 163-165.
Yang, Y., & Land, K. C. (2006). A mixed models approach to the age‐period‐cohort analysis of repeated cross‐section surveys, with an application to data on trends in verbal test scores. Sociological methodology, 36(1), 75-97.
Reither, E. N., Hauser, R. M., & Yang, Y. (2009). Do birth cohorts matter? Age-period-cohort analyses of the obesity epidemic in the United States. Social science & medicine, 69(10), 1439-1448.
Reither, E. N., Masters, R. K., Yang, Y. C., Powers, D. A., Zheng, H., & Land, K. C. (2015). Should age-period-cohort studies return to the methodologies of the 1970s?. Social Science & Medicine, 128, 356-365.
Thanks to Anna Shirokanova for proofreading the post.