We recently wrote an article comparing the conclusions that followed from two different approaches to quantifying the reliability and replicability of psychopathology symptom networks. Two commentaries on the article have raised five core criticisms, which are addressed in this response with supporting evidence. 1) We did not over-generalize about the replicability of symptom networks, but rather focused on interpreting the contradictory conclusions of the two sets of methods we examined. 2) We closely followed established recommendations when estimating and interpreting the networks. 3) We also closely followed the relevant tutorials, and used examples interpreted by experts in the field, to interpret the bootnet and NetworkComparisonTest results. 4) It is possible for statistical control to increase reliability, but that does not appear to be the case here. 5) Distinguishing between statistically significant versus substantive differences makes it clear that the differences between the networks affect the inferences we would make about symptom-level relationships (i.e., the basis of the purported utility of symptom networks). Ultimately, there is an important point of agreement between our article and the commentaries: All of these applied examples of cross-sectional symptom networks are demonstrating unreliable parameter estimates. While the commentaries propose that the resulting differences between networks are not genuine or meaningful because they are not statistically significant, we propose that the unreplicable inferences about the symptom-level relationships of interest fundamentally undermine the utility of the symptom networks.
Bibliographical noteFunding Information:
Funding: This work was supported by a Macquarie University Research Fellowship; Grants L30 MH101760 R01AG053217, and U19AG051426 from the U.S. National Institutes of Health; and by the Templeton Foundation.
© 2021 Taylor & Francis Group, LLC.
Copyright 2021 Elsevier B.V., All rights reserved.
- network analysis
- network theory of mental disorders
- psychopathology symptom networks
- replicability crisis