Christian Welzel addresses my criticism

The following is Christian Welzel’s response he kindly provided following the post in which I summarized my criticism of his and Ronald Inglehart’s paper on measurement invariance:

This dismissive portrayal of our contribution does not come as a surprise. Indeed, no one who considers “internal coherence” as the prime criterion of measurement quality can be happy about our article. But that does not make the uncomfortable truth less true: the logic of internal coherence often prompts scholars to disqualify a multi-item construct as invalid, despite the fact that this very construct shows meaningful, important and powerful “external linkages” with its expected correlates. Scholars who advocate internal coherence as the gold standard of measurement quality offer no explanation of why weak internal coherence coexists so easily with strong external linkages. In fact, most of these scholars do not even address this phenomenon. Or they play it down, as if external linkages had nothing to do with a construct’s reality anchorage.

Our approach, by contrast, addresses directly why weak internal coherence can go together handily with strong external linkages. The concept of “compositional substitutability” provides the answer: two similar total scores on a multi-item construct can derive from very different admixtures of single item scores, but when the total scores nevertheless produce the same outcomes, their different compositions are mutually substitutable.
Of course, compositional substitutability is the anti-thesis of internal coherence. For this reason, the whole idea escapes the comprehension of all those who believe that a construct is valid only if it shows the exact same pattern of inter-item coherence in every different context. For constructs that operate under the rule of compositional substitutability, this notion of measurement equivalence remains flawed. It needs to be replaced by a decidedly consequentialist view that considers measures as equivalent whenever they produce the same outcomes.
Besides, whoever reads our article will quickly recognize that we reason at length about the “realness” of aggregations of individual-level measures. The key point is that aggregations are almost always much more powerfully patterned than the individual-level measures from which they are taken. The reason for this phenomenon is well understood today: aggregation eliminates all of the random measurement error that exists in great abundance at the individual level. Hence, aggregations represent a level of observation that is no less real than the individual level. On the contrary, in terms of linkage patterns, aggregations actually constitute an especially reliable reality level, with demonstrable consequence of great importance.
Finally, the criticism that so called formative constructs are “authoritarian in nature” is the paragon of absurdity. If we took this argument seriously, the obvious conclusion would be to ban all normative theory from science. In this case, medicine would from now on be disallowed to define the concept of health.
I could easily go on here but leave it with this for now. Just one last remark: the allegedly so persuasive disproval of our approach by Boris Sokolov has received a strong response at the very same conference. Here is the link:


Leave a Reply