"There is an old maxim that ‘every model is wrong, but some models are useful’. It takes a lot of work to translate outputs from models to claims about the world. The toolbox of machine learning makes it easier to build models, but it doesn’t necessarily make it easier to extract knowledge about the world, and might well make it harder. As a result, we run the risk of producing more but understanding less.
Science is not merely a collection of facts or findings. Actual scientific progress happens through theories, which explain a collection of findings, and paradigms, which are conceptual tools for understanding and investigating a domain. As we move from findings to theories to paradigms, things get more abstract, broader and less amenable to automation. We suspect that the rapid proliferation of scientific findings based on AI has not accelerated — and might even have inhibited — these higher levels of progress."
https://www.nature.com/articles/d41586-025-01067-2