科技科学

Reply to: Dataset artefacts can partially drive the measured decline in disruption

Nature作者：Michael Park2026年8月12日正文已收录本站

replying to: V. Holst et al. Nature https://doi.org/10.1038/s41586-026-10787-y (2026).

In the accompanying Comment1, Holst et al. claim that the decline in disruptiveness that we documented in Park et al.2 is an artefact of including works that do not cite any references (that is, have zero backward citations). Using the dataset, metric and method advocated by Holst et al.1, we find declines equivalent to benchmark transformations in science. Their own regression model—designed to address their concerns about works with zero references—yields large, significant declines for papers and patents (P < 0.01), a result that is presented in their supplementary tables yet left unaddressed, despite directly contradicting their central claim. Their critique is further undermined by severe quality issues in their data, which contain three times as many works with zero references as our data. We trace this excess to their inclusion of at least 2.8 million editorials, obituaries and comments, 1.5 million books and proceedings and 254,000 product and artistic reviews. Twenty per cent of their sample is non-research content that almost by definition lacks references. Simple keyword searches highlight the problem’s severity, identifying among others 456 For Dummies guides, 50 Dr. Seuss and Curious George books, and the Captain Underpants series, all without references, in their sample. Applying granular document-type classification reveals that non-research content decreased from 40% to 8% of their sample between 1945 and 2010—a shift that is sufficient to generate the decline in works with zero references that they attribute to metadata errors in our study. Standard practice excludes such content to safeguard against the metadata quality concerns at the centre of their critique—concerns that their dataset exemplifies rather than addresses. Their supplementary Web of Science analysis, framed as a direct replication of our study, rests on shifting and impossible accounts of its methodology. Declining disruptiveness has been documented in nearly 100 studies across databases, metrics and non-citation-based measures3. The evidence does not support an artefact-based explanation.

Data availability

Code availability

Our code builds on Funk et al. (https://doi.org/10.5281/zenodo.7258379); additional scripts will be made available upon request.

References

  1. Holst, V., Algaba, A., Tori, F., Wenmackers, S. & Ginis, V. Dataset artefacts can partially drive the measured decline in disruption. Nature https://doi.org/10.1038/s41586-026-10787-y (2026).

    Article  Google Scholar 

  2. Park, M., Leahey, E. & Funk, R. J. Papers and patents are becoming less disruptive over time. Nature 613, 138–144 (2023).

    Article  ADS  CAS  PubMed  Google Scholar 

  3. Wu, X., Wu, L., Park, M., Leahey, E. & Funk, R. J. Is innovation becoming less disruptive? An inventory of the literature. Preprint at https://doi.org/10.48550/arXiv.2602.05140 (2026).

  4. Kelly, B., Papanikolaou, D., Seru, A. & Taddy, M. Measuring technological innovation over the long run. Am. Econ. Rev. Insights 3, 303–320 (2021).

    Article  Google Scholar 

  5. Wang, D. & Barabási, A.-L. The Science of Science (Cambridge Univ. Press, 2021).

  6. Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019).

    Article  ADS  CAS  PubMed  Google Scholar 

  7. Cui, H., Lin, Y., Wu, L. & Evans, J. A. Aging and the narrowing of scientific innovation. Science 392, 588–591 (2026).

    Article  CAS  PubMed  Google Scholar 

  8. Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023).

    Article  ADS  CAS  PubMed  Google Scholar 

  9. Chu, J. S. G. & Evans, J. A. Slowed canonical progress in large fields of science. Proc. Natl Acad. Sci. USA 118, e2021636118 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Boot, A. & Vladimirov, V. Disclosure, patenting, and trade secrecy. J. Account. Res. 63, 5–56 (2025).

    Article  Google Scholar 

  11. Vaughn, C. R. et al. Return of the disruption score: fetal surgery in the spotlight (1975–2021). J. Pediatr. Surg. 60, 162338 (2025).

    Article  PubMed  Google Scholar 

  12. Tang, X., Li, X. & Yi, M. Will affiliation diversity promote the disruptiveness of papers in artificial intelligence? In iConference 2024 Vol. 14597, 407–415 (Springer, 2024).

  13. Ioannidis, J. P. A., Boyack, K. & Wouters, P. F. Citation metrics: a primer on how (not) to normalize. PLoS Biol. 14, e1002542 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  14. Jaffe, A. B. & Trajtenberg, M. Patents, Citations, and Innovations: A Window on the Knowledge Economy (MIT Press, 2002).

  15. Funk, R. J. & Owen-Smith, J. A dynamic network measure of technological change. Manage. Sci. 63, 791–817 (2017).

    Article  Google Scholar 

  16. Chen, T., Kim, C. & Miceli, K. A. The emergence of new knowledge: the case of zero-reference patents. Strateg. Entrep. J. 15, 49–72 (2021).

    Article  Google Scholar 

  17. Yu, X., Rahwan, T. & Jia, T. Knowledge independence breeds disruption but limits recognition. Preprint at https://doi.org/10.48550/arXiv.2504.09589 (2025).

Download references

Acknowledgements

We thank the National Science Foundation (grant no. 1829168, 1932596, 2318172 and 2449660 to R.J.F. and grant no. 1829302 to E.L.), Wellcome Leap Foundation (grants to R.J.F. and M.P.) and Alfred P. Sloan Foundation (grant no. G-2024-25123 to R.J.F.) for financial support of work related to this project. The funders had no role in study design, data collection and analysis or preparation of the manuscript. We thank T. Gebhart, J. Lane, J. Owen-Smith, L. Bornmann, C. Leibel, A. Zaheer, J. Nahm, K. Kedrick, M. VanEseltine, R. Murciano-Goroff, H. Kang, L. Wu, S. Wu, X. Wu, Y. J. Kim, D. Kim and Z. Ge for helpful comments.

Author information

Authors and Affiliations

  1. Organisational Behaviour, INSEAD, Fontainebleau, France

    Michael Park

  2. School of Sociology, University of Arizona, Tucson, AZ, USA

    Erin Leahey

  3. Carlson School of Management, University of Minnesota, Minneapolis, MN, USA

    Russell J. Funk

Authors

  1. Michael Park
  2. Erin Leahey
  3. Russell J. Funk

Contributions

R.J.F. and M.P. designed the analyses. R.J.F. conducted the analyses and wrote the manuscript. M.P. contributed to analyses and manuscript revisions. E.L. contributed to manuscript revisions.

Corresponding author

Correspondence to Russell J. Funk.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data figures and tables

Extended Data Fig. 1 Persistent decline across independently developed disruptiveness measures.

This figure demonstrates that the decline in disruptiveness persists even when papers with zero references are excluded, regardless of disruption metric or dataset. The plots track average (percentile) values of four independently developed measures. Values are plotted separately for Web of Science (left) and SciSciNet (right). The measures include CYG5 (Citation Year Gap), which calculates the average age gap between references cited by citing works relative to the focal paper; Is D5, a binary indicator for whether the CD5 value is positive; CD5noK, which excludes references-only citations (the nK term) from the denominator; and CD55, which introduces a threshold requiring future works to cite multiple references of the focal paper. All measures exclude zero-reference documents and are percentile-normalized to enable comparison across scales. Declines are statistically significant across all measures and datasets (P < 0.001; see Supplementary Information, section 3 and  Supplementary Table 2 for full results). Shaded bands correspond to 95% confidence intervals.

Extended Data Fig. 2 Severe overrepresentation of CD = 1 works in the SciSciNet data of Holst et al.

This figure compares the distribution of CD index values between the Park et al.2 (PLF) datasets (papers in Web of Science and patents in PatentsView) and the Holst et al.1 (HATWG) SciSciNet data. In Park et al.2, the proportion of CD = 1 documents is 4.3% for Web of Science papers and 4.9% for patents. In the SciSciNet data of Holst et al.1, 23.1% of documents have CD = 1—a 5.4-fold overrepresentation. This excess is consistent with the inclusion of non-research content that receives occasional citations but makes none (see main text and Supplementary Information, section 9). CD = 1 works are central to the argument by Holst et al.1 for excluding works with zero references, yet the overrepresentation in their data raises serious concerns about the quality of the dataset underlying their critique.

Extended Data Table 1 Selected zero-backward-citation works in the analytical sample of Holst et al.

Full size table

Supplementary information

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Park, M., Leahey, E. & Funk, R.J. Reply to: Dataset artefacts can partially drive the measured decline in disruption. Nature 656, E14–E21 (2026). https://doi.org/10.1038/s41586-026-10788-x

Download citation

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1038/s41586-026-10788-x