Peter Molenaar’s widely cited paper and a recent Fisher et al. (2018) claim that between-individual differences that are often used to explain within-individual processes cannot be used for this purpose, or at least may invoke a large bias. In some students and researchers, this article created a false impression that between-individual (or cross-sectional) studies are totally useless in arguing about within-individual processes. In this post, I claim that cross-sectional studies aren’t useless and sometimes are the only possible way to find out about within-individual processes.
Imagine a person who has been raised religious, always goes to church every Sunday, and prays every day before sleep. Imagine also that this particular person is also strongly against abortion. A typical longitudinal study would measure her religiosity and her attitudes toward abortion multiple times during, say, five years, and then test if there is a correlation between change in a level of religiosity and change in a level of the attitude. If the person’s religiosity hasn’t changed, the classic longitudinal study would efficiently estimate zero relations between these variables because one of them is constant. The fact that religiosity has been constantly high for five years by no means implies it doesn’t influence attitudes. My point is that the longitudinal study might detect relations between variables that change, and totally useless when it faces no within-individual change. Therefore, within-individual designs aren’t almighty in discovering within-individual processes. In my example, a between-individual design is the only way to find out about what might be going on within an individual. We would see that more religious individuals are less in favor of abortions, and may theorize that a constantly high level of religiosity leads to a constantly negative attitude to abortion.
Following Judea Pearl, in order to make a valid conclusion, we have to overcome a mere observation of associations (which he treats as a lowest level of inference). To be able to make valid inferences, we have to imagine and reason the counterfactuals, i.e. the events that have not happened. In my example, the person’s low level of religiosity is counterfactual, but we can infer what would happen if it were the case – and the only way to do this is to use cross-sectional, between-individual data.
As I noticed above, within-individual designs are limited to features that change. My guess is that more stable features of person and personality (such as values, personality traits, gender, social class) tend to affect behavior and attitudes in much much higher degree than characteristics that change. Indeed, important things don’t change fast, that’s why they are important! Therefore, between-individual studies might well be even more powerful than within-individual studies in discovering and explaining within-individual processes.
These limitations of within-individual designs apply to surveys as well as to experiments; additional limitation of experiments is that we cannot manipulate most of things, and those we actually can aren’t very powerful forces.
Of course, we have to have in mind that between-individual designs describe first of all between-individual differences, and only with some serious assumptions (which we have to explicate and reflect on) they may suggest a course of within-individual processes. The main assumption here is that a sample of individuals represents a sample of states of a single individual. Whether this assumption is reasonable or not is subject to discuss, but we shouldn’t blindly deny the use of cross-sectional designs in studying within-person processes. It might be a substantively driven decision in studies of within-person processes, going beyond organizational concerns.
